Tqdm is a popular pakcage to show progress bar when executing a loop. Here are a few issues and tips when using tqdm.
Windows display issue
On Windows, it seems that tqdm will print a new line when showing the progress, which is annoying.
According to issue here, this is due
to Unicode handling in Windows terminal. We can add
ascii=True in tqdm() to
prevent this behavior. Works for me on Windows for both Cmder and plain cmd.
Iterate over each line in a file and show progress?
When we are handling files with large number of lines, we may want to see the progress. To see the progress, we need to know the number of lines in this file. Here is a sample code to show the progress bar:
from tqdm import tqdm file_path = "test.txt" num_lines = sum(1 for line in open(file_path, 'r')) with open(file_path, 'r') as f: for line in tqdm(f, total=num_lines): # deal with this line... pass
Show progress for concurrent.futures.ThreadPoolExecutor
from concurrent.futures import ThreadPoolExecutor import time from tqdm import tqdm def square(x): time.sleep(0.01) return x*x def main(): nums = list(range(10000)) with ThreadPoolExecutor() as executor: results = list(tqdm(executor.map(square, nums), total=len(nums))) if __name__ == "__main__": main()
The output of
executor.map(square, nums) is a generator containing all the
returned results from the function
square(). So we need to supply the total
number in the
In the latest version of tqdm, we can also simply the above code by using
tqdm.contrib.concurrent.thread_map() method, which is essentially a wrapper
around above code:
from concurrent.futures import ThreadPoolExecutor import time from tqdm.contrib.concurrent import thread_map def square(x): time.sleep(0.01) return x*x def main(): nums = list(range(10000)) with ThreadPoolExecutor() as executor: results = thread_map(square, nums) if __name__ == "__main__": main()
The above code is tested on tqdm version 4.48.2.
License CC BY-NC-ND 4.0