to delete duplicates and check for alive domains only ,etc we can use only the terminal to do this cat resutlts.txt | sort -u > resultst-unique.txt

in this step we now have only unique subdomains so let’s check it alive or not ? cat results-unique.txt | httprobe -c 100 > results-alive.txt