This content comes from @What's worth buying APP, and the views only represent the author's own |Author: Panda is not a cat QAQ
introduction
I believe that many people will use network disks, and at the same time have learned about some "network disk aggregation search engines", such as disk Duoduo; The essence of PanSearch and Cloud Disk Home is to integrate the resource search functions of multiple different network disk services into one platform, which realizes more efficient search through the whole and improves the efficiency and convenience of resource search.
Before its network disk aggregation, it was more of a "seed aggregation search station". Through crawler technology or self-collection and other means, obtain seed resources and include them in the database, and then integrate and classify the database with the help of APIs, and finally search through the front-end web. It's just that the download speed of seeds is affected by the number of seeds, and the sharing spirit of the Internet is getting worse and worse, so the seeds are actually used less and less. Today, Panda wants to introduce the seed search engine and front-end project based on DHT crawler - "Bitmagnet-Next-Web"
deploy
The project needs to use a database, and the project is based on Bitmagnet, so here we need to use Docker Compose deployment, in which some ports, mapping paths and variables need to be changed appropriately according to their own device environment. The sample is as follows:
version: "3"
services:
bitmagnet-next-web:
image: 127.0.0.1:4321/journey0ad/bitmagnet-next-web:latest
container_name: bitmagnet-next-web
ports:
- "3050:3000"
restart: unless-stopped
environment:
- POSTGRES_DB_URL=postgres://postgres:postgres@postgres:5432/bitmagnet
# - POSTGRES_HOST=postgres
# - POSTGRES_PASSWORD=postgres
depends_on:
postgres:
condition: service_healthy
Bit magnet:
image: 127.0.0.1:4321/ghcr.io/bitmagnet-io/bitmagnet:latest
container_name: BitMagnet
ports:
# API and WebUI port:
- "3333:3333"
# BitTorrent ports:
- "3334:3334/tcp"
- "3334:3334/udp"
restart: unless-stopped
environment:
- POSTGRES_HOST=postgres
- POSTGRES_PASSWORD=postgres
# - TMDB_API_KEY=your_api_key
command:
- worker
- run
- --keys=http_server
- --keys=queue_server
# disable the next line to run without DHT crawler
- --keys=dht_crawler
depends_on:
postgres:
condition: service_healthy
postgres:
image: 127.0.0.1:4321/postgres:16-alpine
container_name: bitmagnet-postgres
volumes:
- /volume1/docker/Bitmagnet/postgres:/var/lib/postgresql/data
ports:
- "5431:5432"
restart: unless-stopped
environment:
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=bitmagnet
- PGUSER=postgres
shm_size: 1g
healthcheck:
test:
- CMD-SHELL
- pg_isready
start_period: 20s
interval: 10s
Among them, the keys in the Bitmagnet container need to be filled in by yourself, and DHT also needs to be found by yourself, which is related to the material content of your seed library.
At the same time, due to the removal of domestic image libraries, we need to add the acceleration address before the container name, Panda uses its own docker acceleration here, so it uses 127.0.0.1:4321, which is an intranet address, do not use it directly. The following acceleration address is recommended.
After everything is ready, we open the SSH port of the NAS, connect to the NAS with the SSH tool, enter sudo -i to get the administrator permission, then enter the project pathpoint in the form of the command cd project path, and finally enter docker-compose up -d to start the container.
After all the projects are downloaded and started, the container is successfully deployed.
experience
The project consists of three parts, of which bitmagnet-next-web is the front-end search interface, taking the above yml file of Panda as an example, then the address is NASIP+3050 port number, and you can see such an interface at this time.
You can get the search results by directly typing the resources you want, the search results support size and time sorting, and you can also see the inclusion time, where the inclusion time is the time to crawl the resource after the Bitmagnet container is created, not the seed release time.
Click on the resource to see the resource details, and the magnetic link supports direct copying, and there is a file list and a content preview window below.
In the lower right corner of the Home, you can also see the current number of resources in the seed bank.
The second part of the container is our core part Bitmagnet, which can also be opened through the NASIP+3333 port through the web interface according to the yml file of Panda. The project is used to store seed data and provide indexing, but the interface does not support Chinese. (Because the material involves copyright or prohibition, it has been coded)
Through web page translation, no Chinese has little impact, and at the same time, it is generally directly used for front-end web search, and this interface is almost not used.
Click on the resource to view the file; Edit the tag and delete the torrent, and on the right you can see the file size and the time it was published.
summary
The project is quite good, but as a DHT crawler network, the most important thing is to continue to crawl data, compared to building it yourself, I personally recommend using it directly, after all, other people's databases have accumulated hundreds of millions, and building them yourself needs to accumulate slowly, and many old resources are not easy to find. Finally, thank you to the fan @江不吃吃 for the material provided.
The above is all the content of this issue, if you think it's interesting or helpful to you, you might as well like the collection, and finally hope to get your attention, let's see you next time!
The author declares that there is no interest in this article, and friends are welcome to communicate rationally and discuss harmoniously~