Can the decoder in a transformer model be parallelized like the encoder? As far as I understand the encoder has all the tokens in the sequence to compute the self-attention scores. But for a decoder this is not possible (in both training and testing), as self attention is calculated based on previous timestep outputs. Even if we consider some technique like teacher forcing, where we are concatenating expected output with obtained, this still has a sequential input from the previous timestep. In this case, apart from the improvement in capturing long-term dependencies, is using a transformer-decoder better than say an lstm when comparing purely on the basis of parallelization?
Transformer based decoding
1.1k Views Asked by shiredude95 At
1
There are 1 best solutions below
Related Questions in DEEP-LEARNING
- [Caffe]: Check failed: ShapeEquals(proto) shape mismatch (reshape not set)
- Caffe net.predict() outputs random results (GoogleNet)
- Implementation of convolutional sparse coding in deep networks frameworks
- Matlab example code for deep belief network for classification
- Two errors while running Caffe
- How to speed up caffe classifer in python
- Caffe Framework Runtest Core dumped error
- Scan function from Theano replicates non_sequences shared variables
- Why bad accuracy with neural network?
- Word2Vec Sentiment Classification with R and H2O
- What is gradInput and gradOutput in Torch7's 'nn' package?
- Error while drawing net in Caffe
- How does Caffe determine the number of neurons in each layer?
- Conclusion from PCA of dataset
- Google Deep Dream art: how to pick a layer in a neural network and enhance it
Related Questions in TRANSFORMER-MODEL
- Using parseincludes in Laravel5 Fractal
- how to transform result to map in hibernate5.2
- Cognos Framework manager alternatives on Linux only
- Modifying python AST while preserving comments
- Java Hibernate Transformer AliasToBeanNestedResultTransformer
- How to change color and stroke of one type of edges
- ibm cognos transformer multiple fact table not supported by dimension
- java standard lib produce wrong xml 1.1
- Mule returning a MessageCollection from component
- XLM-RoBERTa token - id relationship
- what's the difference between "self-attention mechanism" and "full-connection" layer?
- Transformer Image captioning model produces just padding rather than a caption
- Using Transformer's decoder to extract sentences
- Is there any way to self create Transformer to run on Coral board?
- Use Asus Transformer Prime as USB Debugger
Related Questions in SEQ2SEQ
- Does using FP16 help accelerate generation? (HuggingFace BART)
- how does nn.embedding for developing an encoder-decoder model works?
- why seq2seq model return negative loss if I used a pre-trained embedding model
- Is there a way for a closed domain chatbot to build using seq2seq, generative modeling or other methods like RNNs?
- Temporal Fusion Transformer model training encountered Gradient Vanishing
- Training a transformer to copy sequence to identical sequence?
- tensorflow multivariable seq 2 seq model return only lagged forcast
- Error: Invalid argument: ConcatOp : Dimensions of inputs should match
- Transforming keras model output during training and use multiple losses
- LSTM seq2seq input and output with different number of time steps
- predict sequence of tuples using Transformer model
- How does the finetune on transformer (t5) work?
- ValueError: Shapes (None, 16) and (None, 16, 16) are incompatible (LSTMs)
- How to translate my own sentence using Attention mechanism?
- how to create a seq2seq NLP model based on a transformer with BERT as the encoder?
Related Questions in ENCODER-DECODER
- An error was encountered when setting batch_size to 1 in sequence_to_sequence_implementation.ipynb（when batch_size > 1,it 's ok）
- Appropriate loss function in pytorch when output is an array of float numbers
- how does nn.embedding for developing an encoder-decoder model works?
- ValueError: Layer lstm_3 expects 35 inputs, but it received 3 input tensors
- Is the encoder in decoder = LSTM(128)(encoder) the hidden state or the input to the decoder?
- Why is my attention model worse than non-attention model
- How to get padding mask for cross attention of decoder of transformer
- Decoder error not supported error when render 360 video on web application
- Problem with Sending Logon Message Using SBE Over a Socket in Java
- creating an encoder-decoder LSTM model for text samarization in tensorflow
- TensorFlow Callback loss monitor not retrieving the loss for training data
- RNN Encoder Decoder Model generates empty Output
- Error while generating dimension of universal sentence encoder embedding
- Encoder-Decoder for Trajectory Prediction
- How to train an encoder-decoder model?
Related Questions in SEQUENCE-MODELING
- Is adding a FC Linear layer on top of seq2seq architecture a potential source of data leaking from future to past?
- how to create train - dev - test sets from a given dataset in sequence models
- How can I do a sequence-to-sequence model (RNN / LSTM) with Keras with fixed length data?
- How to implement Bi-Directional Conv LSTM in Pytorch
- Is there any other reason why we make sequence length the same using padding?
- 1D CNN predictions plot mismatch with actual time series plot
- TF-IDF vector vs a vector of tokens
- How the function nn.LSTM behaves within the batches/ seq_len?
- Transformer based decoding
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
You are correct in that both an LSTM decoder and a Transformer decoder process one token at a time, i.e. they are not parallelized over the output tokens. The original Transformer architecture does not parallelize the decoder; only in the encoder is the sequence of tokens processed in parallel. For a detailed summary of the Transformer architecture and training/testing process you can see this article.