'ner_ontonotes_bert_mult' model Custom train

Question

When I want to train this 'ner_ontonotes_bert_mult' model with my custom dataset it is showing the error below. (I have saved my datset in the ~\.deeppavlov\downloads\ontonotes folder that was mentioned in [deeppavlov documentation][1]. )

PS C:\Users\sghanta\Desktop\NER> & c:/Users/sghanta/Desktop/NER/env/Scripts/Activate.ps1
(env) PS C:\Users\sghanta\Desktop\NER> & c:/Users/sghanta/Desktop/NER/env/Scripts/python.exe c:/Users/sghanta/Desktop/NER/train_model.py
C:\Users\sghanta\Desktop\NER\env\lib\site-packages\numpy\_distributor_init.py:32: UserWarning: loaded more than 1 DLL from .libs:
C:\Users\sghanta\Desktop\NER\env\lib\site-packages\numpy\.libs\libopenblas.PYQHXLVVQ7VESDPUVUADXEVJOBGHJPAY.gfortran-win_amd64.dll
C:\Users\sghanta\Desktop\NER\env\lib\site-packages\numpy\.libs\libopenblas.WCDJNK7YVMPZQ2ME2ZZHJJRJ3JIKNDB7.gfortran-win_amd64.dll
  stacklevel=1)
Traceback (most recent call last):
  File "c:/Users/sghanta/Desktop/NER/train_model.py", line 12, in <module>
    ner_model = train_model(configs.ner.ner_ontonotes_bert_mult)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\deeppavlov\__init__.py", line 29, in train_model
    train_evaluate_model_from_config(config, download=download, recursive=recursive)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\deeppavlov\core\commands\train.py", line 92, in train_evaluate_model_from_config
    data = read_data_by_config(config)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\deeppavlov\core\commands\train.py", line 58, in read_data_by_config
    return reader.read(data_path, **reader_config)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\deeppavlov\dataset_readers\conll2003_reader.py", line 56, in read
    dataset[name] = self.parse_ner_file(file_name)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\deeppavlov\dataset_readers\conll2003_reader.py", line 106, in parse_ner_file
    raise Exception(f"Input is not valid {line}")
Exception: Input is not valid 
 O

(env) PS C:\Users\sghanta\Desktop\NER> 

After cleaning the dataset the above error has gone but this is the new error. New Error

2021-08-12 02:43:35.335 ERROR in 'deeppavlov.core.common.params'['params'] at line 112: Exception in <class 'deeppavlov.models.bert.bert_sequence_tagger.BertSequenceTagger'>
Traceback (most recent call last):
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\client\session.py", line 1365, in _do_call
    return fn(*args)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\client\session.py", line 1350, in _run_fn
    target_list, run_metadata)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\client\session.py", line 1443, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [13,13] rhs shape= [37,37]
         [[{{node save/Assign_76}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\training\saver.py", line 1290, in restore
    {self.saver_def.filename_tensor_name: save_path})
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\client\session.py", line 956, in run
    run_metadata_ptr)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\client\session.py", line 1180, in _run
    feed_dict_tensor, options, run_metadata)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\client\session.py", line 1359, in _do_run
    run_metadata)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\client\session.py", line 1384, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [13,13] rhs shape= [37,37]
         [[node save/Assign_76 (defined at C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\framework\ops.py:1748) ]]

Original stack trace for 'save/Assign_76':
  File "c:/Users/sghanta/Desktop/NER/train_model.py", line 12, in <module>
    ner_model = train_model(configs.ner.ner_ontonotes_bert_mult)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\deeppavlov\__init__.py", line 29, in train_model
    train_evaluate_model_from_config(config, download=download, recursive=recursive)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\deeppavlov\core\commands\train.py", line 121, in train_evaluate_model_from_config
    trainer.train(iterator)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\deeppavlov\core\trainers\nn_trainer.py", line 334, in train
    self.fit_chainer(iterator)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\deeppavlov\core\trainers\fit_trainer.py", line 104, in fit_chainer
    component = from_params(component_config, mode='train')
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\deeppavlov\core\common\params.py", line 106, in from_params
    component = obj(**dict(config_params, **kwargs))
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\deeppavlov\core\models\tf_backend.py", line 76, in __call__
    obj.__init__(*args, **kwargs)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\deeppavlov\core\models\tf_backend.py", line 28, in _wrapped
    return func(*args, **kwargs)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\deeppavlov\models\bert\bert_sequence_tagger.py", line 529, in __init__
    **kwargs)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\deeppavlov\models\bert\bert_sequence_tagger.py", line 259, in __init__
    self.load()
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\deeppavlov\core\models\tf_backend.py", line 28, in _wrapped
    return func(*args, **kwargs)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\deeppavlov\models\bert\bert_sequence_tagger.py", line 457, in load
    return super().load(exclude_scopes=exclude_scopes, **kwargs)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\deeppavlov\core\models\tf_model.py", line 251, in load
    return super().load(exclude_scopes=exclude_scopes, **kwargs)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\deeppavlov\core\models\tf_model.py", line 54, in load
    saver = tf.train.Saver(var_list)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\training\saver.py", line 828, in __init__
    self.build()
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\training\saver.py", line 840, in build
    self._build(self._filename, build_save=True, build_restore=True)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\training\saver.py", line 878, in _build
    build_restore=build_restore)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\training\saver.py", line 508, in _build_internal
    restore_sequentially, reshape)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\training\saver.py", line 350, in _AddRestoreOps
    assign_ops.append(saveable.restore(saveable_tensors, shapes))
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\training\saving\saveable_object_util.py", line 73, in restore
    self.op.get_shape().is_fully_defined())
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\ops\state_ops.py", line 227, in assign
    validate_shape=validate_shape)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\ops\gen_state_ops.py", line 66, in assign
    use_locking=use_locking, name=name)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\framework\op_def_library.py", line 794, in _apply_op_helper
    op_def=op_def)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\util\deprecation.py", line 507, in new_func
    return func(*args, **kwargs)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\framework\ops.py", line 3357, in create_op
    attrs, op_def, compute_device)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\framework\ops.py", line 3426, in _create_op_internal
    op_def=op_def)
  File "C:\Users\sghanta\Desktop\NER\env\lib\site-packages\tensorflow_core\python\framework\ops.py", line 1748, in __init__
    self._traceback = tf_stack.extract_stack()

Can anyone explain how to solve it. [1]: http://docs.deeppavlov.ai/en/master/features/models/ner.html

Answer 1

conll2003_reader dataset reader failed to parse the following line:

 O

conll2003_reader dataset reader expects that line is either empty or contains a token and a label. In your case only label is present.

So, I would suggest to clean your data from empty lines with labels.

Sample text from DeepPavlov docs:

EU B-ORG
rejects O
the O
call O
of O
Germany B-LOC
to O
boycott O
lamb O
from O
Great B-LOC
Britain I-LOC
. O

China B-LOC