Django/FilePond ModelForm

Django Model FileField on ModelForm with FilePond and Django post_save Signal.

Initially File uploads were handled within Django default file storage and served through MEDIA URL, MEDIA ROOT included in static urlpatterns and uploaded during Form POST, which doesnt look good anyway. So its time for some tweaks.

For file uploading through UI we will use FilePond. It is a javascript file upload library that is able to handle async file uploads when the User e.g., drag and drops a file.
For server-side handling we will use django-drf-filepond. It offers a server-side API ready to be used from filepond Clients to handle and store uploads.

Filepond setup

One of the easy ways to setup Filepond is through CDN but it offers multiple options.
Check FilePond Installation

To straight up use Filepond through CDN we include the following in the scripts of head tag prior to the body tag.

  <link href="https://unpkg.com/filepond@^4/dist/filepond.css" rel="stylesheet" />  

  <script src="https://unpkg.com/filepond@^4/dist/filepond.js"></script>
  <script src="https://unpkg.com/filepond-plugin-file-validate-type/dist/filepond-plugin-file-validate-type.js"></script>

We use an extra Filepond plugin here to validate files upon insert which we will use to accept only pdf.
You can find more useful plugins FilePond plugins.

Filepond options setup

<script type="text/javascript">
    FilePond.registerPlugin(FilePondPluginFileValidateType);
    FilePond.setOptions({
        acceptedFileTypes: ["application/pdf"],
        chunkUploads: true,
        chunkSize: 500000,
        server: {
            url: '/applications/fp', //TODO: change to match drf url
            headers: {'X-CSRFToken': '{{ csrf_token }}'},
            process: {
                url: '/process/',
                method: 'POST',
                headers: {
                'X-CSRFToken': $('input[name="csrfmiddlewaretoken"]').val()
                },
                withCredentials: false,
                ondata: (formData) => {
                var upload_field = '';
                for (var item of formData.keys()) {
                    upload_field = item;
                    break;
                }
                if (upload_field !== '') {
                    formData.append('fp_upload_field',upload_field);
                }
                return formData;
                },
                onerror: (response) => { console.log(response.data); },
            },
            patch: '/patch/',
            revert: '/revert/',
            fetch: '/fetch/?target='
        }
    });
    FilePond.parse(document.body);
</script>

• First we register the FilePondPluginFileValidateType and set in options the acceptedFileTypes attribute as application/pdf.
• Secondly we setup four URLs served from django-drf-filepond extension which our Filepond client will use. process, patch, revert, fetch
• FilePond will parse the document for inpult fields including filepond in their class.
• To render the fields correctly Filepond expects for input fields to have the name attribute setup as filepond. That is ok if there is only one FileField but in case of handling many different files in form POST you may need to have different names on each one.
  To use any name attribute you like you need to include the code handling ondata: formData above. The above javascript will inject the extra form field ‘fp_upload_field’ with the value of the FilePond file field name. Issue.

Server-side django-drf-filepond setup

django-drf-filepond Installation

  pip install django-drf-filepond

settings.py:

...
INSTALLED_APPS = [
    ...
    'django_drf_filepond',
    ...
]

DJANGO_DRF_FILEPOND_UPLOAD_TMP = os.path.join(BASE_DIR, 'filepond-temp-uploads')
DJANGO_DRF_FILEPOND_STORAGES_BACKEND = 'storages.backends.ftp.FTPStorage'
...

django-drf-filepond uses a temporary local directory to store initially the files that a user adds to the form. Later we can instruct django to store the uploads permantly using the storage defined in DJANGO_DRF_FILEPOND_STORAGES_BACKEND. It supports django-storages so that can be anything you setup for your project e.g., S3. For now we use just an FTP server. Each storage comes with additional variables you need to setup in your settings.py. As for FTP storage we need FTP_STORAGE_LOCATION set with the ftp url.

urls.py:

...
urlpatterns = [
    ...
    re_path(r'^fp/', include('django_drf_filepond.urls'), name='fp'),
    ...
]
...

We added the server-side APIs provided by django-drf-filepond into an existing django app named applications. That is why in setOptions in javascript section above we used applications/fp url as server url. Remember to replace applications with your own app name.

Models and migrations

django-drf-filepond adds database tables to track temporary and permanent stored files. These tables are modelled to TemporaryUpload, StoredUpload Models.

from django_drf_filepond.models import TemporaryUpload, StoredUpload

Both use as primary key a unique upload_id that FilePond Clients put in filefied form data when a File is added on the form by the User. We will make use of this upload_id to handle uploaded files.

FileField to StorageUpload relationship.

#Before
file = models.FileField(upload_to=xxxx, verbose_name="xxxx")

#after
file = models.OneToOneField(StoredUpload, on_delete=models.CASCADE)

Our Model before used a regular FileField offered by Django with upload_to argument to provide the save path appended to default storage location and then served from MEDIA_URL, MEDIA_ROOT paths defined in settings.py.
Now we want a relationship in our Model to the permanently stored file so will use a One-to-One relationship to StoredUpload.
In StoredUpload table each file will have each own record with each own upload_id primary key. So any file uploaded can belong only to one application record.

Now we can run makemigrations and migrate* for django ORM to realize our changes to the database.

Django Signals to permanently store the file.

Once the user adds a file to the form, Filepond will asynchronouslly store the file into the temporary directory we have set in settings.py. We will use a built-in Django signal post_save to watch the TemporaryUpload Model, for new records and once they arrive, we will provide the primary key to django-drf-filepond method store_upload which will move the file from the temporary directory to the permanent storage solution we defined at DJANGO_DRF_FILEPOND_STORAGES_BACKEND in settings.py. This will take place before the User submits the form and after he just has added the file.

Signal setup:
applications/signals.py in our django app applications

from django.db.models.signals import post_save
from django.dispatch import receiver
from django_drf_filepond.models import TemporaryUpload
from django_drf_filepond.api import store_upload
import os
 
@receiver(post_save, sender=TemporaryUpload, dispatch_uid="store_upload") 
def create_profile(sender, instance, created, **kwargs):
    if created:
        #print("TemporaryUpload created, storing to FTP server")
        su = store_upload(instance.upload_id, os.path.join(instance.upload_id, instance.upload_name))

store_upload takes two arguments:

• the upload_id assigned from django_drf_filepond as primary key to TemporaryUpload record which will be the same primary key on StoreUpload record.
• the destination path

ModelForm and request.POST

class UploadDocumentForm(ModelForm):

    class Meta:
        model = ApplicationForm
        fields = ['subfields', 'file']
    
    #render the fileField as a TextInput to receive the filepond upload_id
    def __init__(self, *args, **kwargs):
        current_user = kwargs.pop('current_user')
        super(UploadDocumentForm, self).__init__(*args, **kwargs)
        self.fields['file'].widget = forms.TextInput(attrs={
            'name': 'file',
            })
        self.fields['file'].label = 'XXXX'

When User uploads a file, in the page. Server will return and inject into the bounded form a hidden input with the unique upload_id in place of the file input. So instead of file in POST data, we have to catch that upload_id. Hence, in rendering the ModelForm, we override the widget of the FileField to be render as a TextInput. In the view we will manually assign the file at the form.instance with the StoredUpload object retrieved with that upload_id.

In the view handling the form POST:

    if request.method == 'POST':
        form = UploadDocumentForm(request.POST, current_user = request.user)
        form.instance.foreas = request.user
        filepond_id = request.POST['file']
        form.instance.file = StoredUpload.objects.get(upload_id=filepond_id)
        if form.is_valid():
            form.save()
            return HttpResponseRedirect(reverse('esyd_for_foreas'))

request.POST[‘file’] now is a string with the upload_ui used as primary key in TemporaryUpload and now in StoredUpload referrence records of the uploaded file. We assign to the Model referred at form.instance the file to the StoredUpload object adhering to the OneToOne relationship we defined in the beginning as in the ORM this will be attributed to the specific Model object.

Now we have a working form submitting succesfully while the file upload was managed outsite the form POST request asynchronously to the form submission.

Screenshots - Click to view gallery

Link to the PR with all the changes made: Github