wget –spider –no-check-certificate –force-html -r -l2 https://website.of.interest.com 2>&1 | grep ‘^–‘ | awk ‘{ print $3 }’ | grep -v ‘\.\(css\|js\|png\|gif\|jpg\)$’ >> url-list.txt which will give something like: https://website.of.interest.com/en/Category:BRA https://website.of.interest.com/en/Category:Free_software…

Read More Get url’s from website with wget

Uncategorized

Jeg hadde to git repo’er på samme maskin, men i ulike kataloger. Det som var ønskelig var å merge disse to repo’ene sammen til ett, slik at all historie ble…

Read More Merge to git repo

Git

For several days our Apache webserver went down. Stopped. No html document for you! CPU at 100%, sorry mate! Our server is running Apache web-server as frontend with mod_jk to…

Read More Web site ripping

Apache Firewall

If you have a file with several lines that contains email addresses that you like to extract, you can do that with Linux grep command: File “emails.txt”: somestreng anotherstring 342345…

Read More Match email address

bash Linux