Input in Puppeteer: How to Enter, Retrieve or Clear Values?

Interacting with text fields

Text fields, used for general data entry, are often located via CSS selectors or XPath queries. Puppeteer’s .type() method is the primary tool for inputting text into these fields. You can also simulate keyboard events like pressing Enter or Tab using .keyboard.press().

const textField = await page.$('#textInput');
await textField.type('Your text here');
await textField.keyboard.press('Enter');

Interacting with number inputs

Similar to text fields, number inputs can be targeted and filled using the .type() method. Ensure that you are inputting numerical values as shown below:

const numberInput = await page.$('#numberInput');
await numberInput.type('123');

Interacting with checkboxes

Checkboxes are toggled by using the .click() method on the checkbox element.

const checkbox = await page.$('#checkbox');
await checkbox.click();

const checkboxes = await page.$$('input[type="checkbox"]'); // Selecting all checkboxes
for (const checkbox of checkboxes) {
  await checkbox.click(); // Toggling each checkbox
}

• page.$$('input[type="checkbox"]') selects all checkbox elements on the page.
• The for loop iterates through each checkbox element.
• await checkbox.click() toggles each checkbox.

Interacting with dropdowns

Dropdowns or select elements are navigated using .select() choosing an option based on its value or visible text.

const selectElement = await page.$('select#dropdown');
await selectElement.select('optionValue');

const dropdowns = await page.$$('select'); // Selecting all dropdowns on the page
for (const dropdown of dropdowns) {
  await dropdown.select('optionValue'); // Setting the same value in each dropdown
}

• page.$$('select') identifies and selects all dropdown elements present on the page.
• The loop, traversing through each dropdown element, executes the action.
• await dropdown.select('optionValue') assigns the specified value to each dropdown within the loop.

Interacting with text areas

Text areas, used for larger text inputs, are interacted with similarly to text fields:

const textArea = await page.$('#textAreaInput');
await textArea.type('Your lengthy text here');

Advanced setup cases

Let's cover some advanced setup cases where you need to wait for other elements to load up or to simulate realistic behavior.

1. Waiting for user input with page.waitForSelector()

Puppeteer’s page.waitForSelector() function is an invaluable tool for waiting until a specific element matching the provided selector appears in the DOM. This is particularly useful when waiting for user input or for certain input fields to become available before proceeding with automation.

await page.waitForSelector('#inputField', { visible: true });

2. Simulating realistic user behavior for anti-scraping measures

To bypass anti-scraping mechanisms that detect and block automated activity, Puppeteer allows you to mimic human-like behavior. This involves imitating human interaction patterns such as mouse movements, pauses between actions, and irregular typing speeds.

async function simulateHumanBehavior() {
 // Randomized typing speed
 const typeDelay = Math.floor(Math.random() * 50) + 50;

 // Simulating mouse movement
 await page.mouse.move(100, 200, { steps: 5 });

 // Simulating delayed typing
 await page.type('#usernameInput', 'username', { delay: typeDelay });
 await page.waitForTimeout(1000); // Waiting for 1 second

}

Retrieving values from text inputs

Text inputs often store valuable information that needs extraction. Using page.$eval() or page.evaluate(), you can access and retrieve the value of text inputs.

const textValue = await page.$eval('#textInput', input => input.value);
console.log('Text Input Value:', textValue);

Retrieving values from number inputs

Similarly, for number inputs, you can use the same approach to access and retrieve their values

const numberValue = await page.$eval('#numberInput', input => input.value);
console.log('Number Input Value:', numberValue);

Retrieving values from checkboxes

Checkboxes provide their state (checked or unchecked) through the checked property. You can retrieve this property to determine the checkbox’s current state.

const isChecked = await page.$eval('#checkbox', checkbox => checkbox.checked);
console.log('Checkbox State:', isChecked);

const checkboxes = await page.$$('input[type="checkbox"]'); // Selecting all checkboxes
for (const checkbox of checkboxes) {
 const isChecked = await checkbox.evaluate(input => input.checked);
 console.log('Checkbox State:', isChecked);
}

• page.$$('input[type="checkbox"]') selects all checkbox elements on the page.
• The for loop traverses each checkbox element.
• await checkbox.evaluate(input => input.checked) retrieves the checked state of each checkbox.

Retrieving values from dropdowns

Dropdowns or select elements store selected values. You can use the select element’s value property to retrieve the chosen option.

const selectValue = await page.$eval('select#dropdown', select => select.value);
console.log('Selected Dropdown Value:', selectValue);

const dropdowns = await page.$$('select'); // Selecting all dropdowns on the page
for (const dropdown of dropdowns) {
 const selectValue = await dropdown.evaluate(select => select.value);
 console.log('Selected Dropdown Value:', selectValue);
}

• page.$$('select') selects all dropdown elements on the page.
• The for loop iterates through each dropdown element.
• await dropdown.evaluate(select => select.value) retrieves the value of each dropdown.

Retrieving values from text areas

Text areas, used for larger text inputs, can be retrieved similarly to text fields as shown below.

const textAreaValue = await page.$eval('#textAreaInput', textarea => textarea.value);
console.log('Text Area Value:', textAreaValue);

Clearing number input values

To clear a number input’s value, you can set its value attribute to an empty string.

const numberInput = await page.$('#numberInput'); // Selecting the number input by its selector
await numberInput.evaluate(input => input.value = ''); // Setting the value to an empty string to clear it

Clearing checkbox state

To uncheck a checkbox using Puppeteer, you can directly simulate a click on the checkbox element.

const checkbox = await page.$('#checkbox');
const isChecked = await checkbox.evaluate(input => input.checked);

if (isChecked) {
  await checkbox.click(); // Toggles checkbox state to unchecked
}

• page.$('#checkbox') selects the checkbox element using its selector.
• const isChecked retrieves the current state of the checkbox and if (isChecked) checks if the checkbox is already checked.
• If the checkbox is checked, await checkbox.click() simulates a click action on the checkbox, toggling its state from checked to unchecked.

const checkboxes = await page.$$('input[type="checkbox"]');
for (const checkbox of checkboxes) {
  const isChecked = await checkbox.evaluate(input => input.checked);

  if (isChecked) {
    await checkbox.click(); // Unchecks the checkbox if it's checked
  }
}

• page.$$() selects all checkbox elements on the page.

The loop iterates through each checkbox element and if the checkbox is checked, await checkbox.click() simulates a click to uncheck it.When dealing with multiple checkboxes that need to be cleared, you can loop through them and uncheck each one individually.

Clearing dropdown selection

Clearing a dropdown selection involves resetting it to its default or empty state.

const dropdown = await page.$('#dropdown'); // Selecting the dropdown by its selector
await dropdown.select(''); // Selecting an empty option to clear the dropdown selection

• page.$('#dropdown') selects the dropdown by its selector.
• await dropdown.select('') uses select() to choose an empty option (if available) within the dropdown, effectively clearing the selection.

If the dropdown does not have an empty default option, you might need to select a specific default option by value or index to clear the selection.When dealing with multiple dropdowns that need their selection cleared, you can iterate through each dropdown element and reset their values.

const dropdowns = await page.$$('select'); // Selecting all dropdowns on the page
for (const dropdown of dropdowns) {
  await dropdown.select(''); // Selecting an empty option for each dropdown to clear selections
}

• page.$$('select') selects all dropdown elements on the page.
• The for loop iterates through each dropdown element.
• await dropdown.select('') selects an empty or default option for each dropdown, effectively clearing their selections.

Clearing text area values

Clearing values from text areas involves a similar approach to text inputs as shown below:

const textArea = await page.$('#textAreaInput');
await textArea.click({ clickCount: 3 }); // Selecting all text in the text area
await textArea.type(String.fromCharCode(8)); // Pressing backspace to delete

• page.$('#textAreaInput') selects the text area element.
• await textArea.click({ clickCount: 3 }) simulates a triple click to select all text.
• await textArea.type(String.fromCharCode(8)) simulates pressing the backspace key, deleting the selected text.