Web Scraping Crypto Prices With Python
This is the most beautiful soup
--
Recently I was looking for some data from a website to do my own analysis on cryptocurrencies. I believe there’s probably going to be a large bull run coming soon and I wanted to do my own analysis to capture gainz when the time comes.
As always, if you want to do good analysis, you need good data. For cryptocurrencies, look no further than CoinMarketCap.
You don’t always need APIs
Traditionally we’d like to use an API to fetch data, but we all know that APIs are designed with limitations for the free user. The free plan of most APIs are just there to tease you enough for you to pull out your wallet and spend money on a paid plan. Personally, I’m cheap so I’m looking to stay in the free realm.
What I first did was go through CoinMarketCap’s website and look for their API and pricing plans to see if the free plan could get the job done.
This is what I see:
In comes the Soup
The free API is not going to give us what we want but don’t fret, we’re going to use a very VERY handy workaround. Assuming you have python installed on your machine, here’s what we’re going to do:
We’re installing BeautifulSoup, a popular tool used to web scrape via python. It’s a very easy and simple to use tool and I’ll show you exactly how to use it to do something that CoinMarketCap’s free API won’t do for us.
Though we’re making a Beautiful Soup, we’re going to need some other key ingredients, and I’ll show you why as we go along.
Now we’re going to want to play around and do some exploration. We’ll do a simple get request of the main page into the BeautifulSoup object.
This will give us the entire web page, and we can specify different parts of the page like this:
Fetching Data
Now that we know how get data from a website, it’s time to identify what we need and how to extract it. My intention is to…
