How to set the set_stream_blob_threshold in FIrebird fdb python library?

312 Views Asked by At

Trying to migrate data from Firebird DB to MS Sql Server using fdb(2.0.1) and pyodbc. Since there are blobs in the Firebird database which are over 64K, they are being returned as BlobReader objects. Since i would like not to deal with the bytes myself and write them using pyodbc. The docs say that you can turn off the 64K threshold by passing -1 to the cursor.set_stream_blob_threshold. However that doesn't seem to work, since fdb.fbcore.ProgrammingError is thrown...

https://fdb.readthedocs.io/en/v2.0/reference.html#fdb.Cursor.set_stream_blob_treshold

Here is how i call the function:

import fdb

class Firebird:

    def __init__(self, db_name: str):
        self.__fb_conn = fdb.connect(database=db_name, user='someuser', password='somepass', charset='ISO8859_1')
        self.__fb_cursor = self.__fb_conn.cursor()

        #change the blob safety threshold to unlimited for troubleshooting
        self.__fb_cursor.set_stream_blob_treshold(-1) #doesn't work :(

Here is a stack trace for the error:

(.venv) >python3.8.exe -i
Python 3.8.5 (tags/v3.8.5:580fbb0, Jul 20 2020, 15:57:54) [MSC v.1924 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from commonlibs import Firebird
>>>
>>> fb = Firebird('somedb.fdb')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\user1\dev\commonlibs\Firebird.py", line 13, in __init__
    self.__fb_cursor.set_stream_blob_treshold(int(-1)) #doesn't work :(
  File "C:\Users\user1\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\fdb\fbcore.py", line 3930, in set_stream_blob_treshold
    raise ProgrammingError
fdb.fbcore.ProgrammingError

Per Mark's comment: I don't know much about the data source and what sort of blobs. It was one of those situations where the other teams guy said: "Hey, here is some data from this partner, let's see what inside"

However when trying to pass the obj.read() value to the pyodbc for BlobReader objects, it did insert some of the blobs. However with a lot of them pyodbc would report this error:

 pyodbc.Error: ('HY000', '[HY000] [Microsoft][ODBC SQL Server Driver]Warning: Partial insert/update. The insert/update of a text or image column(s) did not succeed. (0) (SQLPutData); [HY000] [Microsoft][ODBC SQL Server Driver][SQL Server]The text, ntext, or image pointer value conflicts with the column name specified. (7125)')

I was kind hoping i could avoid all this pyodbc and .read() stuff by setting that threshold, but i wonder if the pyodbc error would show up regardless...

0

There are 0 best solutions below