I want to ask about TiDB index design. It is noted in the docs that using auto-increment will cause a hotspot in the write since range-based sharding is used. Is this the same case for secondary indexes? Because secondary indexes are also just another key-value pair in TiKV. If so, for example, would an index on updated_at field (which is monotonically increasing) cause hotspots?
For TiDB, does incremental value in secondary index cause hotspots too?
52 Views Asked by Tetsuya3850 At
1
There are 1 best solutions below
Related Questions in TIDB
- Add additional fields to Linq group by
- couldn't copy pdb file to another directory while consuming wcf web service
- Why are the aliases for string and object in lowercase?
- WPF MessageBox Cancel checkbox check
- Resolve object using DI container with object instance
- Creating a parametrized field name for a SELECT clause
- Does compiler optimize operation on const variable and literal const number?
- Get data from one form to another form in C#
- Writing/Overwriting to specific XML file from ASP.NET code behind
- Deleting Orphans with Fluent NHibernate
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular # Hahtags
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
Yes, in TiDB, the incremental value in a secondary index can also cause hotspots. This is because secondary indexes in TiDB are also stored as key-value pairs in TiKV, and they follow the same range-based sharding mechanism as the primary index [1].
When a large amount of data is written to a table with a secondary index, and the values in the secondary index are continuously increasing (such as the
updated_atfield you mentioned), it can create hotspots in a few regions. These hotspots occur because the data with consecutive index values is written to a few specific regions, which becomes a bottleneck for the entire system [1].To avoid hotspots caused by secondary indexes, you can consider the following strategies:
Randomize the values: If possible, you can introduce randomness to the values in the secondary index. For example, you can add a random prefix or suffix to the
updated_atfield value before inserting it into the secondary index. This can help distribute the data more evenly across different regions.Use a composite index: Instead of creating a secondary index on a single monotonically increasing field, you can create a composite index that includes multiple fields. By including other fields that have more diverse values, you can reduce the likelihood of hotspots.
Adjust the shard size: You can adjust the shard size of the TiKV regions to distribute the data more evenly. However, this requires careful consideration and testing, as it can have an impact on the overall performance and resource utilization of the TiDB cluster.
It's important to note that while these strategies can help mitigate hotspots caused by secondary indexes, they may introduce trade-offs in terms of query performance and index size. It's recommended to carefully evaluate your specific use case and workload characteristics to determine the most suitable index design and optimization strategy.
I hope this answers your question. If you have any further inquiries, please feel free to ask.
[1]: TiDB Documentation - TiDB Best Practices