Tqdm is a popular package to show a progress bar. Here are a few tips when using tqdm.
Windows display issue
On Windows, it seems that tqdm will print a new line when showing the progress, which is annoying.
According to issue here, this is due
to buggy Unicode handling in Windows terminal. We can add
tqdm() to prevent this behavior. Works for me on Windows for both Cmder and
Iterate over each line in a file and show progress?
When we are processing files with large number of lines, we may want to see the progress. To see the progress, we need to know the number of lines in this file. Here is a sample code to show the progress bar using tqdm:
from tqdm import tqdm file_path = "test.txt" num_lines = sum(1 for line in open(file_path, 'r')) with open(file_path, 'r') as f: for line in tqdm(f, total=num_lines): # deal with this line... pass
Show progress for concurrent.futures.ThreadPoolExecutor
Here is an example of using tqdm together with concurrent.futures:
from concurrent.futures import ThreadPoolExecutor import time from tqdm import tqdm def square(x): time.sleep(0.01) return x*x def main(): nums = list(range(10000)) with ThreadPoolExecutor() as executor: results = list(tqdm(executor.map(square, nums), total=len(nums))) if __name__ == "__main__": main()
The output of
executor.map(square, nums) is a generator containing all the
returned results from the function
square(). So we need to supply the total
number in the
In the latest version of tqdm, we can also simplify the above code by using
tqdm.contrib.concurrent.thread_map() method, which is essentially a wrapper
around above code:
from concurrent.futures import ThreadPoolExecutor import time from tqdm.contrib.concurrent import thread_map def square(x): time.sleep(0.01) return x*x def main(): nums = list(range(10000)) with ThreadPoolExecutor() as executor: results = thread_map(square, nums) if __name__ == "__main__": main()
The above code is tested on tqdm version 4.48.2.
License CC BY-NC-ND 4.0