value of configuration option upload.max is not considered in submission form file upload

[saschaszott]

Bug Description

The DS 7 documentation (https://wiki.lyrasis.org/display/DSDOC7x/Submission+User+Interface) contains a description of the (backend) configuration option upload.max

maxSize: Optionally, you can specify a maximum size of file accepted by this UploadConfiguration. If unspecified, default is to use "upload.max" in dspace.cfg/local.cfg, or have no maximum.

We've included upload.max = 2000 (assuming that the value has to be given in bytes) in local.cfg. The submission configuration (access-conditions.xml as well as item-submission.xml) does not contain maxSize.

The DS frontend does not prevent the upload of a 10 MB file via the upload step in the submission form. Furthermore, the DS backend does not validate the size of the incoming file.

[saschaszott]

Furthermore, the setting

<property name="maxSize" value="200"/>

in access-conditions.xml is not considered in the submission form file upload step.

[floriangantner]

We were struggling with the same Issue and solved it in DSpace-Cris by reading the configuration property directly, but using the access-condition is some smoother way. We want to share our experience with file limitations:

On the backend the limitation could be checked by extending the UploadStep or by extending your own UploadableStep implementation and refusing the transmitted item from being stored in the repository.

On the frontend the ng2-file-upload uploader also supports some maxFileSize property which can emit some Upload error when the dropped item's size is greater than the limit and show some notification. The currently maxSize property in config-submission-upload.model

dspace-angular/src/app/core/config/models/config-submission-upload.model.ts

Line 31 in a9c58a1

maxSize: number;

is never used. This property could be used for the uploadFilesOptions. I testwise explored necessary changes on some branch https://github.com/uniba-ub/dspace-angular/tree/test-issue-2848 but this needs further investigation.

These leads to further questions:
The uploader is used on two places:

• in mydspace the normal way is drop the file, than select the collection.
• Should the general maxSize configuration be applied here? Should only the collections be selectable where the belonging limit in the uploadconfiguration is matched? Or should the submission be started with the file ignoring the limitation and show some additional hint in the upload section?

In submission forms via one upload-section:

• assuming only one upload step per submission.
• the access-conditions are only loaded in the upload-section-file, but should be requested in the submission-form-component to initialize the uploader correctly
• case: changing in-progress-submission between collections where the new collection has some smaller maxSize than the collection before?
• case: reducing the maxSize for some existing uploadConfiguration? What happens to existing in-progress-submissions?

[bram-atmire]

I looked into this on current main while working on DSpace/DSpace#13106, and can confirm the state @saschaszott and @floriangantner describe, with one addition on what protects the server today:

• upload.max is not set anywhere by default, UploadConfiguration.getMaxSize() returns 0 in that case, and the UI never reads the maxSize it receives, so there is no client-side check at all.
• The backend does not enforce maxSize/upload.max either. The only limit that applies is spring.servlet.multipart.max-file-size (512 MB by default), which Tomcat and Spring enforce while parsing the multipart body. So a multi-GB file is not accepted in full, but up to 512 MB per attempt are transferred and written to temporary storage before the 413 appears, and the user only learns about it at the end.

PRs, following the approach on the exploratory branch mentioned above:

• Expose and enforce the submission upload size limit DSpace#13107: /api/config/submissionuploads reports the effective limit (configured maxSize capped by the multipart limit, absent when unlimited), UploadStep enforces the configured limit with the section error error.validation.filesize, and upload.max is no longer cached permanently.
• Reject files above the upload size limit before sending any data #6248: the uploader refuses larger files when they are added, before any data is sent, and the notification names the limit; the submission form feeds the section's maxSize into the uploader; a server-side 413 is recognised in the submission form and on MyDSpace.
• Document the maxSize attribute of submission upload configurations RestContract#383: documents maxSize (the page showed max-size).

The MyDSpace drop zone still has no client-side check, because the collection and therefore the upload configuration are not known before the file is dropped; that is the open question from the second comment. The broader change, rejecting before any byte reaches the server and uploading in bounded chunks, is what DSpace/DSpace#13106 proposes.