scrape

06 October 2020

Views: 66

Hi there,

I have a very straightforward website scraping task. I need a freelancer to scrape information for each of the 4 university websites (links below):

Duke: http://soc.siss.duke.edu/psp/CSSOC01/EMPLOYEE/SA/c/COMMUNITY_ACCESS.SSS_BROWSE_CATLG.GBL?

USC: https://classes.usc.edu/term-20203/

Stanford: https://explorecourses.stanford.edu/

Northwestern: https://catalogs.northwestern.edu/undergraduate/courses-az/

I need 4 csv files (1 for each university), with 3 columns in the format:
[CHEM] [100-0] [Introduction to Calculus and Chemistry] for Northwestern
[AAAS] [89S] [First-Year Seminar] for Duke
[EARTH] [42] [Moving and Shaking in the Bay Area] for Stanford
[ACMD] [501] [Introduction to Academic Medicine Worldwide] for USC

Scraping Northwestern, Duke, and USC should be straightforward. Be sure to scrape 'All programs' for USC, 'expand all' for Duke, and all the programs for Northwestern.

Stanford is a little more tricky, you just need to be sure to use Selenium and click 'Autumn' (instead of Winter which is default), and scrape the entire page(s).

I'd estimate scraping all 4 will not take more than ~2 hours of coding.

Thank you so much for your time and consideration :)

*****************************
jQuery('[id*="COURSE_LIST"]')

jQuery('table[id*="COURSE_LIST"] tr').each(function(inx,ele){
ele
})

Share