Using mechanicalsoup to set value of form element w/o a name

Question

I have searched through all mechanicalsoup & beautifulsoup documentation but can't figure out how to set the value of a form element using 'id' (because it doesn't have a name).

import mechanicalsoup
browser = mechanicalsoup.StatefulBrowser()

browser.open(my_url)

form = browser.select_form('form[id="login-form"]')

browser.get_current_form().print_summary()
userid = browser.get_current_page().find('input', id='text-userid')
form.set("text-userid", "user")

This gets me -

<input class="login-text-box" id="text-userid" placeholder="Email" type="text" value=""/>
<input class="login-text-box" id="text-password" placeholder="Password" type="password" value=""/>
<input id="button-login" type="submit" value="Sign In"/>
<input id="remember-me-checkbox" name="rememberme" type="checkbox" value="rememberme"/>
LinkNotFoundError: No valid element named text-userid

I have tried referring to the element via id, CSS selector, and other combinations but get the same error. I am able to get the actual tag (which I found as a solution to a similar problem with submit buttons) with:

userid = browser.get_current_page().find('input', id='text-userid')

but form.set() doesn't accept tags directly.

Thanks for any help!

Answer 1

After more research, I learned I should be using Selenium for this case (not mechanicalsoup). I used this guide which did exactly what I needed on the first try.

http://stanford.edu/~mgorkove/cgi-bin/rpython_tutorials/Scraping_a_Webpage_Rendered_by_Javascript_Using_Python.php

Answer 2

Without a name attribute, the input will be quite useless when submitting the form, since the form data will be computed based on this name attribute. Most likely this input will be used by JavaScript, not by form submission, and MechanicalSoup won't help you with JavaScript. See:

https://mechanicalsoup.readthedocs.io/en/stable/faq.html#form-submission-has-no-effect-or-fails

In short: you probably want to use Selenium instead.