I am attempting to create a Bitcoin handle database utilizing btcrecover and Google BigQuery knowledge. Whereas the Ethereum database works advantageous, my Bitcoin database fails to return any addresses, despite the fact that check addresses and random addresses from the blockchain exist within the dataset.
Right here’s what I attempted:
1.Created the database utilizing:
python create-address-db.py --inputlistfile C:UserstestDesktopbtc-addresses-db-20250816 --dbfilename btc-addresses-db-20250816.db --dblength 31
2.Checked addresses utilizing:
python check-address-db.py --dbfilename btc-addresses-db-20250816.db --checkaddresslist ./addressdb-checklists/BTC.txt
python check-address-db.py --dbfilename btc-addresses-db-20250816.db --checkaddresses bc1qxy2kgdygjrsqtzq2n0yrf2493p83kkfjhx0wlh
Observations:
- Ethereum DB works with –dblength 29.
- After I create the Bitcoin DB with –dblength 31 utilizing the complete dataset (~2000 BigQuery recordsdata, 16 GB DB), it finds no addresses.
- A smaller subset (4 BigQuery recordsdata out of 2000) works accurately, with each –dblength 27 (1GB DB) and –dblength 31 (16GB DB).
- Splitting the dataset into halves or subsets results in inconsistent outcomes: some handle ranges are discovered, others aren’t, even inside the identical database.
- Tried a number of Python variations (3.9–3.13) and btcrecover variations (1.6.0, 1.12.0, grasp) with the identical outcomes.
- 64-bit system, all dependencies put in, Rust put in, digital environments used.
- Official Bitcoin database from the creator (addresses-BTC-2011-to-2021-03-31.zip) works accurately.
Instance of inconsistent outcomes from subsets:
| File Vary | Addresses Discovered? |
|---|---|
| 500–520 | Sure |
| 500–599 | No |
| 500–570 | No |
| 550–570 | No |
| 550–559 | Sure |
| 560–569 | Sure |
| 570–579 | Sure |
| 563–579 | Sure |
As you’ll be able to see, some databases efficiently returned addresses, whereas others didn’t. Initially, I believed the issue is likely to be attributable to file 570. Nevertheless, in a subsequent check, the database that included file 570 labored accurately.
In one other check, I created a database from recordsdata 550–570 and examined addresses from every file as samples for this database. The outcomes had been as follows:
- Pattern handle from file 570: discovered
- Pattern handle from file 553: not discovered
- Pattern handle from file 563: discovered
So even inside the identical database, some addresses had been efficiently retrieved whereas others weren’t. This sample means that the difficulty isn’t merely with a single file, however is likely to be associated to how sure addresses are listed or saved within the database.
I’ve tried this on each a Home windows server and my very own PC (with 32GB of
RAM). I even rebuilt the complete database a number of occasions and verified the
hashes afterward to test for any storage or corruption points, each
database recordsdata had similar hashes, so it doesn’t look like a saving
or file corruption drawback. It’s value mentioning that I constructed the
Ethereum database utilizing the identical course of, and that one works completely
advantageous, the difficulty solely occurs with the Bitcoin database.
Query:
Has anybody skilled related points creating massive Bitcoin databases with btcrecover? Might this be a dataset concern, a bug in btcrecover, or an issue with how addresses are listed when –dblength is massive?
