ImportError: cannot import name 'SecretField' from 'pydantic'

Question

I'm using prefect on a gitlab CI and it was working fine until now when it had this error:

from pydantic import BaseModel, Field, SecretField
ImportError: cannot import name 'SecretField' from 'pydantic' (/usr/local/lib/python3.8/site-packages/pydantic/__init__.py)

I tought it was coming form a dependency issu but apparently not, I have compatible version (prefect==2.8.3 and pydantic==1.10.11).

Could come from another packages dependencies ? Here is my requirements.txt file I'm using to generate my docker image:

pandas==1.5.3

asyncpg==0.26.0
prefect==2.8.3
pydantic==1.10.11

simple_salesforce==1.12.3

snowflake-connector-python==3.0.3
snowflake-sqlalchemy==1.4.7

dbt-snowflake==1.4.2

beautifulsoup4==4.12.2

Where It could come from ?

Answer 1

SecretField is not a class in pydantic according to https://docs.pydantic.dev/1.10/usage/types/#secret-types. Perhaps, you want to use SecretStr or SecretBytes classes as shown below.

from pydantic import BaseModel, SecretStr, SecretBytes, ValidationError


class SimpleModel(BaseModel):
    password: SecretStr
    password_bytes: SecretBytes


sm = SimpleModel(password='IAmSensitive', password_bytes=b'IAmSensitiveBytes')

# Standard access methods will not display the secret
print(sm)
#> password=SecretStr('**********') password_bytes=SecretBytes(b'**********')
print(sm.password)
#> **********
print(sm.dict())
"""
{
    'password': SecretStr('**********'),
    'password_bytes': SecretBytes(b'**********'),
}
"""
print(sm.json())
#> {"password": "**********", "password_bytes": "**********"}

# Use get_secret_value method to see the secret's content.
print(sm.password.get_secret_value())
#> IAmSensitive
print(sm.password_bytes.get_secret_value())
#> b'IAmSensitiveBytes'

try:
    SimpleModel(password=[1, 2, 3], password_bytes=[1, 2, 3])
except ValidationError as e:
    print(e)
    """
    2 validation errors for SimpleModel
    password
      str type expected (type=type_error.str)
    password_bytes
      byte type expected (type=type_error.bytes)
    """


# If you want the secret to be dumped as plain-text using the json method,
# you can use json_encoders in the Config class.
class SimpleModelDumpable(BaseModel):
    password: SecretStr
    password_bytes: SecretBytes

    class Config:
        json_encoders = {
            SecretStr: lambda v: v.get_secret_value() if v else None,
            SecretBytes: lambda v: v.get_secret_value() if v else None,
        }


sm2 = SimpleModelDumpable(
    password='IAmSensitive', password_bytes=b'IAmSensitiveBytes'
)

# Standard access methods will not display the secret
print(sm2)
#> password=SecretStr('**********') password_bytes=SecretBytes(b'**********')
print(sm2.password)
#> **********
print(sm2.dict())
"""
{
    'password': SecretStr('**********'),
    'password_bytes': SecretBytes(b'**********'),
}
"""

# But the json method will
print(sm2.json())
#> {"password": "IAmSensitive", "password_bytes": "IAmSensitiveBytes"}

Answer 2

If you can upgrade Prefect and its dependencies, that should fix it. If using pip, run pip install -U prefect

Recent Prefect versions specify a pinned Pydantic version, which should resolve the issue.