i'm having memory issues because looks django loading objects memory when using delete(). there way prevent django doing that?
from django docs:
django needs fetch objects memory send signals , handle cascades. however, if there no cascades , no signals, django may take fast-path , delete objects without fetching memory. large deletes can result in reduced memory usage. amount of executed queries can reduced, too.
https://docs.djangoproject.com/en/1.8/ref/models/querysets/#delete
i don't use signals. have foreign keys on model i'm trying delete, don't see why django need load objects memory. looks does, because memory rising query runs.
you can use function iterate on huge number of objects without using memory:
import gc def queryset_iterator(qs, batchsize = 500, gc_collect = true): iterator = qs.values_list('pk', flat=true).order_by('pk').distinct().iterator() eof = false while not eof: primary_key_buffer = [] try: while len(primary_key_buffer) < batchsize: primary_key_buffer.append(iterator.next()) except stopiteration: eof = true obj in qs.filter(pk__in=primary_key_buffer).order_by('pk').iterator(): yield obj if gc_collect: gc.collect() then can use function iterate on objects delete:
for obj in queryset_iterator(hugequeryset.objects.all()): obj.delete() for more information can check this blog post.
Comments
Post a Comment