Skip to content

Latest commit

 

History

18 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Current Local That Works database scraper

This aims to scrape this page (embedded here) to turn it into a CSV.

Uses https://mircozeiss.com/json2csv/ for creating the CSV.

Issues and time-tracking

Running this

  1. Clone this repo
  2. Run make
  3. In the browser window opened by the scrape target, open your browser's dev tools.
  4. Find the console log item that is the contents of a CSV. Save it.

Other make targets:

  • make clean removes all files generated by make
  • make download redownloads the database
  • make injected inserts our scraping JS into the downloaded page and performs some text replacements that are undone by injection.js
  • make scrape opens the post-injection page in the browser

Assumptions

  • you're on a Mac with all the Homebrew dependencies, including make and open
    • Linux users may be able to alias open=xdg-open

About

A scraper for https://current.org/LTWdatabase

Resources

Code of conduct

Contributing

Stars

0 stars

Watchers

2 watching

Forks

Releases

Sponsor this project

Packages

Contributors

Languages

Sponsor
SponsoredKunjungi sekarang
Promo