Can you firstly scrape the SP500 list from this wiki page? This will be our stock universe. I won't mind if you do it through Jupyter or .py format, preferably .py format.
https:
en.wikipedia.org/wiki/List_of_S%26P_500_companies
The S&P 500 stock market index is maintained by S&P Dow Jones Indices. It comprises 503 common stocks which are issued by 500 large-cap companies traded on American stock exchanges (including the 30 companies that compose the Dow Jones Industrial Average). The index includes about 80 percent of the American equity market by capitalization. It is weighted by free-float market capitalization, so more valuable companies account for relatively more weight in the index. The index constituents and the constituent weights are updated regularly using rules published by S&P Dow Jones Indices. Although called the S&P 500, the index contains 503 stocks because it includes two share classes of stock from 3 of its component companies.
For all the development work, let's structure in this way:
1. In a Notebook, put all the comments/mark downs, intermediate codes, test cases, analysis, etc. there and share with me in html format for review/comments, etc.
2. After we finish each part, save the code in a .py file, and make them callable with proper documentation
3. Structure the codes/files properly