: Effective upload to backend? Let's say I have a website with frontend and backend. The website allows users to upload some data ranging from hundreds of MB to GB. How do you effectively upload

@Rivera981

I definitely do not recommend encoding the uploads and storing them in a relational database, that's just introducing a lot of overhead and complexity for no gain. File systems were designed for files :)

You definitely want to design this to be flexible, because today you might use your own server, tomorrow AWS, and months from now a CDN. Or any combination of the above. To that effect, try:


Have uploads post to upload.yourdomain.com where the minimum needed to check auth / permissions and the upload itself will run.
Create static.yourdomain.com to use for linking to these uploads in order to serve them to browsers.


These could, at least originally, all reside on the same server. The point is, you have the flexibility of having the upload sub domain send files to a storage network, large RAID array, content delivery network or whatever else you come up with. This sub domain could also handle jobs like virus scanning, re-sizing, encoding, or whatever other processing has to happen when an upload completes.

The static domain is then easy to change as you change your infrastructure, it's job is to point to the correct location of the processed uploads and ensure that the proper headers for a static server are set.

For access, just use a *.yourdomain.com cookie and you're done - no need to worry about third party cookies, certificate complexity (one wildcard is fine) and other quirks associated with spreading this out.

This of course makes more or less sense depending on what kinds of uploads you're dealing with, but the general idea should remain pretty much the same. Just think a bit about the TTLs you set for the DNS-A records for both sub domains. Don't set them over 4 hours until you've settled on a rather permanent setup, and quite possibly much lower until you have.

Vote Up Vote Down