colly golang save local copy

252 Views Asked by At

I'm writing Scraping in Golang and wanted to implement this logic, first I save HTML copy on my computer and then I process this copy, after reading the documentation I found the method response.Save, but in the end it saves the result of the work. Can anyone tell me how to do it correctly?

    package main
  
    import (
        "fmt"
        "github.com/gocolly/colly"
    )
    
    func main() {
        GetPage := colly.NewCollector()
        GetPage.Clone()
        GetPage.OnHTML("a[href]", func(htmlElement *colly.HTMLElement) {
            link := htmlElement.Attr("href")
            fmt.Printf("Link: %q -> %s/n", htmlElement.Text, link)
            GetPage.Visit(htmlElement.Request.AbsoluteURL(link))
        })
        GetPage.OnRequest(func(request *colly.Request) {
            fmt.Println("Visiting", request.URL.String())
        })
        GetPage.OnResponse(func(response *colly.Response) {
            response.Save("1.html")
        })
        GetPage.Clone()
        GetPage.Visit("https://mail.com") ```
0

There are 0 best solutions below