Hi guys i could use some help.
I am currently working on a project where we allow our users to create let's say papers, where they can create multiple entries for the given paper.
The project currently consists of a svelte frontend, kotlin backend and s3 storage.
In the database I have a table "paper" and a table "paper_entries". The "paper_entries" have a column "content" which is a field they can edit in the frontend with a content editor "tiptap" and is saved as HTML.
The HTML is sanatized with "jsoup" on the backend before it's palced in the DB. So far so good.
At this point I would like to allow the users to handle images. So I created an extension for handling the upload with presigned urls, which the frontend requests from the backend, uploads the image, updates the content with a tag and sends it to the backend.
- FE request upload url to s3 from BE
- uploades image to s3
- FE sends updated content with to BE
*id = object name on s3
Now the big struggle begins.... As a paper can have multiple entries (lets assume 10-100) and every entry can have X images, i have to generate the signed urls for every entry. For the replacement i use a regex to find the target ID then generate the signed url and set it as src. ()
This is where I noticed quite a performance impact for retrieving the data. From ~30ms to ~600ms. The replacement was done in the BE.
Because of this performance impact I now tried to replace the data on the client side (browser) which is much nicer because the user does not really see a big delay because the content is a bit further down on the page.
But because of this client side replacement I must make a call for each image to the backend to get the signed url. Which works but it still feels kind a horrible to do.
Are there any nicer solutions? I did google a lot to find some similar conditions but couldn't.
So in short I tried:
- Replace the content in BE
Quite huge performance impact: From ~30ms to ~600ms. (Maybe i did something terribly wrong???)
Regex I used: /<img\W?(src=\".*\")?(id=\"([a-zA-Z0-9-]+)\"\Wwidth=\"\d+\"\W?height=\"\d+\")\W?>/gm
- Replace content in FE
Needs to make a be call for every image to get the signed url.
- Tried to create the signed url in front end
So far I'm kind a stuck here because the libs out there like AWS and minio are not really tools to use in the browser.
late reply, but might be useful for future googlers.
Since s3 only supports presigned url for specific objects, you could look into serving content through cloudfront. Cloudfront allows you to generate presigned access to an entire folder, so you can give access to specific folders within the s3 bucket.
With this method, you wont need to find all img tags to replace with a presigned url because the presigned token is stored set in the cookie of your response. Cloudfront checks your cookies for this token and allows retrieval of the file/object. https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-signed-cookies.html
It is also usually better to place cloudfront infront of your s3 because it serves content from cached edge and cloudfront gives free 1tb bandwidth a month. Cloudfront also has WAF, so theres quite a few benefits from using s3.