简单站长图片异步爬虫

以下是一个简单的Python示例，使用aiohttp库编写的图片异步爬虫框架：

import asyncio
import aiohttp
import os
 
async def download_image(url, session, directory):
    async with session.get(url) as response:
        if response.status == 200:
            file_name = os.path.basename(url)
            with open(os.path.join(directory, file_name), 'wb') as f:
                while True:
                    chunk = await response.content.read(1024)
                    if not chunk:
                        break
                    f.write(chunk)
 
async def main(urls, directory='images'):
    async with aiohttp.ClientSession() as session:
        tasks = [download_image(url, session, directory) for url in urls]
        await asyncio.gather(*tasks)
 
# 使用方法:
# urls = ['http://example.com/image1.jpg', 'http://example.com/image2.jpg']
# asyncio.run(main(urls))

这段代码定义了一个download_image异步函数，它使用aiohttp会话异步下载图片。main函数则是异步运行多个下载任务的入口点。你需要提供一个图片链接列表urls，然后调用asyncio.run(main(urls))来运行爬虫。这个简单的例子演示了如何使用异步I/O操作来提高效率，并减少对服务器的请求压力。