爬虫案例:彼岸图网4K图片数据爬取
以下是一个简化的版本,用于演示如何使用Python爬取彼岸图网上的4K高清图片。
import requests
from bs4 import BeautifulSoup
import os
def download_image(url, filename):
response = requests.get(url)
with open(filename, 'wb') as file:
file.write(response.content)
def crawl_behance_images(username, max_pages=2):
base_url = f'https://www.behance.net/{username}'
os.makedirs(username, exist_ok=True)
page_number = 1
while page_number <= max_pages:
print(f'Crawling page {page_number}')
url = f'{base_url}/gallery/{{project_id}}/revisions?page={page_number}&display_type=grid&sort=recent'
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
image_urls = [image['src'] for image in soup.find_all('img', class_='image') if image['src'].endswith('.jpg')]
for image_url in image_urls:
filename = os.path.basename(image_url)
download_image(image_url, os.path.join(username, filename))
page_number += 1
crawl_behance_images('username_example', max_pages=1) # 替换为具体的用户名
请注意,由于版权和平台政策,未经允许,不应在未经允许的情况下爬取大量图片。此代码仅用于学习目的,并假定用户已获得合法权限。
评论已关闭