I'm working on a React-based front-end application with a Java backend that includes a video labeling feature. Our application is encountering significant performance issues during a specific workflow involving bounding box annotations on video frames. I'm seeking insights or suggestions to address this bottleneck.
Environment
Front-end: React (using the React Player library) Back-end: Java Video Source: AWS S3 API: Custom GetMedia API through API Gateway
Workflow and Issue
Our application allows users to annotate video frames with bounding boxes.
The typical workflow involves:
- Pausing the video.
- Drawing a bounding box on the current frame.
- Moving to the next frame (or skipping 10 frames ahead) using a 'next' button.
- Repeating the annotation process for subsequent frames.
- After repeating this process for around 100 frames (or 10 cycles of 10 frames), when trying to navigate back or change frames, the loading becomes exceedingly slow.
API Details
- API Call: GetMedia (custom streaming API).
- Parameters: MediaID, User, UriInfo, AuthToken, TransitiveAuthToken, MediaContentType, Range.
- Data Source: Media content is fetched from S3.
Observations
- The issue is primarily observed after the bounding box annotation process only for certain videos and that too of length > 2 min.
- Multiple GET requests are made per frame switch for the streaming API via react-player.
- Delays are notable in 'waiting for server response' and 'content download'. Increasing exponentially upto > 1 min.
- There is not delay in shift of bounding boxes if I move to an older already annotated frame. The delay is only in video switching back to that frame.
Attempted Solutions
- Monitored memory and resource usage without identifying any leaks or excessive usage.