scrapy 2.3 自定義圖像管道示例

下面是圖像管道的完整示例，其方法如上圖所示：

import scrapy
from itemadapter import ItemAdapter
from scrapy.exceptions import DropItem
from scrapy.pipelines.images import ImagesPipeline

class MyImagesPipeline(ImagesPipeline):

    def get_media_requests(self, item, info):
        for image_url in item['image_urls']:
            yield scrapy.Request(image_url)

    def item_completed(self, results, item, info):
        image_paths = [x['path'] for ok, x in results if ok]
        if not image_paths:
            raise DropItem("Item contains no images")
        adapter = ItemAdapter(item)
        adapter['image_paths'] = image_paths
        return item

要啟用自定義媒體管道組件，必須將其類導(dǎo)入路徑添加到 ?ITEM_PIPELINES? 設(shè)置，如以下示例中所示：

ITEM_PIPELINES = {
    'myproject.pipelines.MyImagesPipeline': 300
}