51 lines
2.1 KiB
Plaintext
51 lines
2.1 KiB
Plaintext
This program opens a range of datastores between two specified dates and extracts all the data
|
|
for a set of user-defined publishers.
|
|
|
|
Add the gitea repo (this has to be done just once):
|
|
dotnet nuget add source --name Gitea https://www.gitea.econlabs.org/api/packages/markusmobiuspublic/nuget/index.json
|
|
|
|
Clone repo:
|
|
git clone https://www.gitea.econlabs.org/markusmobiuspublic/RetroIndex
|
|
|
|
The navigate to RetroIndex folder:
|
|
cd RetroSampleApp
|
|
dotnet publish
|
|
cd bin\Release\net9.0\publish
|
|
dotnet RetroSampleApp.dll
|
|
|
|
|
|
Expected Output
|
|
-------------
|
|
|
|
7 stores are available between 6/15/2025 12:00:00 AM and 6/22/2025 12:00:00 AM:
|
|
first available time stub: 2025_06_16_00_00_00
|
|
last available time stub: 2025_06_22_00_00_00
|
|
Opening datastore 2025_06_16_00_00_00 from disk ...
|
|
Opening datastore 2025_06_17_00_00_00 from disk ...
|
|
Opening datastore 2025_06_18_00_00_00 from disk ...
|
|
Opening datastore 2025_06_19_00_00_00 from disk ...
|
|
Opening datastore 2025_06_20_00_00_00 from disk ...
|
|
Opening datastore 2025_06_21_00_00_00 from disk ...
|
|
Opening datastore 2025_06_22_00_00_00 from disk ...
|
|
publishers: 8594
|
|
nytimes.com: exists in data
|
|
washingtonpost.com: exists in data
|
|
Reading filtered publishers ..
|
|
-----------------------
|
|
Publisher: nytimes.com
|
|
7/7 days: ["2025_06_16_00_00_00","2025_06_17_00_00_00","2025_06_18_00_00_00","2025_06_19_00_00_00","2025_06_20_00_00_00","2025_06_21_00_00_00","2025_06_22_00_00_00"]
|
|
total pages: 36046
|
|
pages with publish data between 6/15/2025 12:00:00 AM and 6/22/2025 12:00:00 AM: 604
|
|
-----------------------
|
|
Publisher: washingtonpost.com
|
|
7/7 days: ["2025_06_16_00_00_00","2025_06_17_00_00_00","2025_06_18_00_00_00","2025_06_19_00_00_00","2025_06_20_00_00_00","2025_06_21_00_00_00","2025_06_22_00_00_00"]
|
|
total pages: 19560
|
|
pages with publish data between 6/15/2025 12:00:00 AM and 6/22/2025 12:00:00 AM: 828
|
|
-----------------------
|
|
|
|
|
|
In order to incorporate in your own project make sure you reference the BingScrapeReadLib (2.0.0+) nuget package:
|
|
https://www.gitea.econlabs.org/markusmobiuspublic/-/packages/nuget/bingscrapereadlib/2.0.0
|
|
|
|
The RetroSampleApp demonstrates how.
|