Python check file size of during download

Find centralized, trusted content and collaborate around the technologies you use most. Connect and share knowledge within a single location that is structured and easy to search. The problem is that i may stumble upon url that is a download of a really large file and I am not interseted in downloading it.

I found this question - Link - and it suggests using urllib and urlopen. I don't want to contact the server twice. If the server supplies a Content-Length header, then you can use that to determine if you'd like to continue downloading the remainder of the body or not.

If the server does not provide the header, then you'll need to stream the response until you decide you no longer want to continue. To do this, you'll need to make sure that you're not preloading the full response. Stack Overflow for Teams — Collaborate and share knowledge with a private group. Create a free Team What is Teams? Collectives on Stack Overflow. Learn more. Asked 5 years ago. Active 5 years ago. Viewed 3k times. I am crawling the web using urllib3. I want to limit the file size to 25MB.

It works OK, but I can't figure how to get the file size before download to compare if it was updated on the server or not. Can this be done as if I was downloading the file from a FTP server?

Jon: thank for your quick answer. It works, but the filesize on the web server is slightly less than the filesize of the downloaded file. Using the returned-urllib-object method info , you can get various information on the retrieved document. Example of grabbing the current Google logo:.

It's a dict, so to get the size of the file, you do urllibobject. Using these will take advantage of the webserver's caching rules and will return a Not Modified status code if the content hasn't changed.

The size on disk sometimes is different from the actual size of the data. It depends on the underlying file-system and how it operates on data. When a file is written on the disk, it is stored in a 'sort-of linked list' of disk blocks.

When a certain block is used to store part of a file, no other file contents will be stored in the same blok, so even if the chunk is no occupuing the entire block space, the block is rendered unusable by other files. Example: When the filesystem is divided on b blocks, and we need to store b file, two blocks will be occupied.

The first block will be fully utilized, while the second block will have only 88b utilized and the remaining b will be unusable resulting in 'file-size-on-disk' being b. This is why Windows has different notations for 'file size' and 'size on disk'. Stack Overflow for Teams — Collaborate and share knowledge with a private group. Create a free Team What is Teams?

Collectives on Stack Overflow. Learn more. Get size of a file before downloading in Python Ask Question. Asked 13 years, 3 months ago. Active 28 days ago. Viewed 50k times. If you are building project where user needs to upload any file, you can check the file size.

If the file size is zero, you can prompt user to upload valid file. You can also check if the file is greater than some threshold value. Save my name, email, and website in this browser for the next time I comment. What do you want to Learn Today? How to get the File Size in Python Program? Code Python. Syntax os.

Aniruddha Chaudhari.