sql - How to prevent django from loading objects in memory when using `delete()`? -


i'm having memory issues because looks django loading objects memory when using delete(). there way prevent django doing that?

from django docs:

django needs fetch objects memory send signals , handle cascades. however, if there no cascades , no signals, django may take fast-path , delete objects without fetching memory. large deletes can result in reduced memory usage. amount of executed queries can reduced, too.

https://docs.djangoproject.com/en/1.8/ref/models/querysets/#delete

i don't use signals. have foreign keys on model i'm trying delete, don't see why django need load objects memory. looks does, because memory rising query runs.

you can use function iterate on huge number of objects without using memory:

import gc  def queryset_iterator(qs, batchsize = 500, gc_collect = true):     iterator = qs.values_list('pk', flat=true).order_by('pk').distinct().iterator()     eof = false     while not eof:         primary_key_buffer = []         try:             while len(primary_key_buffer) < batchsize:                 primary_key_buffer.append(iterator.next())         except stopiteration:             eof = true         obj in qs.filter(pk__in=primary_key_buffer).order_by('pk').iterator():             yield obj         if gc_collect:             gc.collect() 

then can use function iterate on objects delete:

for obj in queryset_iterator(hugequeryset.objects.all()):     obj.delete() 

for more information can check this blog post.


Comments