Website Name: Blue Cross and Blue Shield of Texas This folder has the data for the link: https://www.cigna.com/legal/compliance/machine-readable-files The index json file which contains links to downloadable json.gz is: 2022-10-21_Blue-Cross-and-Blue-Shield-of-New-Mexico_index.json ---------------------------------------------Detials---------------------------------------------------------------------- reporting_entity_type: Health Insurance Issuer reporting_entity_name: Cigna Health Life Insurance Company in_network 4356 links for were found in 2022-12-01_cigna-health-life-insurance-company_index.json Allowed_amount 22505 links for were found in 2022-12-01_cigna-health-life-insurance-company_index.json As these links are all not have a .json.gz ending, instead some links leads to other webpages with nothing to download, so we will filter the links from which we can get the data. All the downloadable links are also written to this file: 2022-10-21_Blue-Cross-and-Blue-Shield-of-New-Mexico_index.txt All the downloaded file are extracted from .json.gz to .json and then converted into .csv. After conversion the .json files are deleted to save memory. --------------------------------------------------------------------------------------------------------------------------- The .csv files are the actual data ! for the codes either check out the main.py or final_script.ipynb