according to the official documentation , the maximum input tokens for a PaLM 2 for Text (text-unicorn) model is 8192. However I am getting a 'token limit error' when submitting prompts larger than 7565 tokens (approximately, could be slightly more). I have verified this by counting the prompt tokens before submitting them using their official token counting API. Am I missing anything here? What is the reason for this?
Real max tokens count for google vertex ai text-unicorn model
159 Views Asked by DarioB At
1
There are 1 best solutions below
Related Questions in GOOGLE-CLOUD-PLATFORM
- Why do I need to wait to reaccess to Firestore database even though it has already done before?
- Unable to call datastore using GCP service account key json
- Troubleshooting Airflow Task Failures: Slack Notification Timeout
- GoogleCloud Error: Not Found The requested URL was not found on this server
- Kubernetes cluster on GCE connection refused error
- Best way to upload images to Google Cloud Storage?
- Permission 'storage.buckets.get' denied on resource (or it may not exist)
- Google Datastream errors on larger MySQL tables
- Can anyone explain the output of apache-beam streaming pipeline with Fixed Window of 60 seconds?
- Parametrizing backend in terraform on gcp
- Nonsense error using a Python Google Cloud Function
- Unable to deploy to GAE from Github Actions
- Assigned A record for Subdomain in Cloud DNS to Compute Engine VM instance but not propagated/resolved yet
- Task failure in DataprocCreateClusterOperator when i add metadata
- How can I get the long running operation with google.api_core.operations_v1.AbstractOperationsClient
Related Questions in GOOGLE-CLOUD-VERTEX-AI
- Fine tuning code-bison model on Google Vertex AI
- Custom Service Account with KFP pipelines in Vertex AI
- Setting up Vertex AI SDK for Python
- GCP Vertex AI Pipelines static outound IP address
- How can we disable the "Open Jupyterlab" button (proxy) on GCP Vertex AI Workbenches?
- Streaming responses from Vertex AI endpoints
- GCP Colab Enterprise shared VPC connection
- Google vertex endpoint is unavilable when deploying new model
- VertexAI serving_container_predict_route for custom container returns 404
- Vertex AI Models Throws AttributeError: After Langchain 0.0295 Version
- Gives INVALID_ARGUMENT error when pushing a new unstructured data into datastore
- Impossibility of creating a service control in Vertex AI Search
- Passing parameters to Llama 2 deployed in Vertex AI
- Custom Labels in Vertex AI Pipeline PipelineJobSchedule
- Get the health status for a Google Vertex Endpoint
Related Questions in GOOGLE-CLOUD-AI
- Real max tokens count for google vertex ai text-unicorn model
- Does Gemini embedding model support languages other than English?
- Impossible to activate Duet AI despite receiving provisioning email
- google-cloud/aiplatform vertex AI predictionserviceclient truncated response NodeJS
- Google Document AI Api integration in ReactJs
- Accessing data in Google Cloud Storage without downloading it
- Format issue when calling Vertex AI Custom Job Endpoint
- Why does this fail with "'async for' requires an object with __aiter__ method, got coroutine"
- Trying to get a Vertex AI prediction in BigQuery using Python
- Is TabNet's mask explanation biased?
- `steps_per_epoch` in google ai platform multi-worker distributed training
- Is there a way to have users access my Google Cloud AI Platform Notebook Instance easily?
- SQL Server : DATETIME to INT
- No such object error when saving TensorFlow model trained on Google Cloud AI Platform to a Google Cloud Storage Bucket
- Vertex AI Custom Container Training Job python SDK - InvalidArgument 400 error
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
It's indeed odd that there's a discrepancy between the official document and the actual token count, the only reason that I can think of is buffers. The maximum token input written in the document might be referring to the overall system capacity but the actual usable limit appears to be lower.
You can submit a ticket in our Issue tracking system and product feature requests so the rightful team can look it up and provide clarifications on the document.