Markus Mobius 6797924508 Fixes
2023-07-25 11:18:38 -04:00
2023-07-25 01:57:00 -04:00
2023-07-25 01:57:00 -04:00
2023-07-25 01:57:00 -04:00
2023-07-25 11:18:38 -04:00
2023-07-25 01:57:00 -04:00

This program opens a range of datastores between two specified dates and extracts all the data
for a set of user-defined publishers.

Add the gitea repo (this has to be done just once):
dotnet nuget add source --name Gitea https://www.gitea.econlabs.org/api/packages/markusmobiuspublic/nuget/index.json

Clone repo:
git clone https://www.gitea.econlabs.org/markusmobiuspublic/RetroIndex

The navigate to RetroIndex folder:
cd RetroSampleApp
dotnet publish
cd bin\Debug\net7.0\publish
dotnet RetroSampleApp.dll


Expected Output
-------------

7 stores are available between 2/15/2023 12:00:00 AM and 2/21/2023 12:00:00 AM:
first available time stub: 2023_02_15_00_00_00
 last available time stub: 2023_02_21_00_00_00
Opening datastore 2023_02_15_00_00_00 from disk ...
Opening datastore 2023_02_16_00_00_00 from disk ...
Opening datastore 2023_02_17_00_00_00 from disk ...
Opening datastore 2023_02_18_00_00_00 from disk ...
Opening datastore 2023_02_19_00_00_00 from disk ...
Opening datastore 2023_02_20_00_00_00 from disk ...
Opening datastore 2023_02_21_00_00_00 from disk ...
publishers: 9020
nytimes.com: exists in data
washingtonpost.com: exists in data
Reading filtered publishers ..
-----------------------
Publisher: nytimes.com
7/7 days: ["2023_02_15_00_00_00","2023_02_16_00_00_00","2023_02_17_00_00_00","2023_02_18_00_00_00","2023_02_19_00_00_00","2023_02_20_00_00_00","2023_02_21_00_00_00"]
total pages: 105223
pages with publish data between 2/15/2023 12:00:00 AM and 2/21/2023 12:00:00 AM: 17030
-----------------------
Publisher: washingtonpost.com
7/7 days: ["2023_02_15_00_00_00","2023_02_16_00_00_00","2023_02_17_00_00_00","2023_02_18_00_00_00","2023_02_19_00_00_00","2023_02_20_00_00_00","2023_02_21_00_00_00"]
total pages: 28779
pages with publish data between 2/15/2023 12:00:00 AM and 2/21/2023 12:00:00 AM: 4143
-----------------------


In order to incorporate in your own project make sure you reference the BingScrapeReadLib (1.5.0+) nuget package:
https://www.gitea.econlabs.org/markusmobiuspublic/-/packages/nuget/bingscrapereadlib/1.5.0

The RetroSampleApp demonstrates how.
Description
This library opens a range of datastores between two specified dates and extracts all the data for a set of user-defined publishers.
Readme 88 KiB
Languages
C# 100%