I am trying to build a model in TensorFlow that should be able to take in 2 input strings and return a resulting string, an example is given below:
Example:
Input 1: "Learning from failure is a valuable lesson."
Input 2: "Substitute 'valuable lesson' with 'opportunity for growth and learning'."
Output: "Learning from failure is an opportunity for growth and learning."
So far in my model I am Tokenizing, padding and encoding input 1 and input 2 separately, what is the best way of dealing with this situation, should I contaminate them and treat as a single input or have them separately as I do right now. It was hard to find any helpful examples online, if anyone has any helpful guides that would be much appreciated.
Setup of my encoders:
# Source encoder
sourceInput = Input(shape=(sourceSequencesTokenizedMaxLength,))
sourceVocabSize = len(sourceTokenizer.word_index) + 1
sourceEmbedding = Embedding(sourceVocabSize, EncoderEmbeddingDim)(sourceInput)
sourceEncoderLSTM = LSTM(EncoderLTSMUnits, return_state=True)
sourceEncoderOutputs, sourceStateH, sourceStateC = sourceEncoderLSTM(sourceEmbedding)
sourceInitialState = [sourceStateH, sourceStateC]
# Change encoder
changeInput = Input(shape=(changeSequencesTokenizedMaxLength,))
changeVocabSize = len(changeTokenizer.word_index) + 1
changeEmbedding = Embedding(changeVocabSize, EncoderEmbeddingDim)(changeInput)
changeEncoderLSTM = LSTM(EncoderLTSMUnits, return_state=True)
changeEncoderOutputs, changeStateH, changeStateC = changeEncoderLSTM(changeEmbedding)
changeInitialState = [changeStateH, changeStateC]
Below is the half setup of my decoder, still trying to fugure out what to pass for the initial_state:
# Decoder
decoderInput = Input(shape=(targetSequencesTokenizedMaxLength,))
targetVocabSize = len(targetTokenizer.word_index) + 1
decoderEmbedding = Embedding(targetVocabSize, DecderEmbeddingDim)(decoderInput)
decoderLstm = LSTM(DecderLTSMUnits, return_sequences=True, return_state=True)
decoderOutputs, _, _ = decoderLstm(decoderEmbedding, initial_state= NOT_IMPLEMENTED)