Problem request timeout for looping with CURL

Question

I have script like this :

function getNewProductToEcommerce($merk, $shopID){
    // assuming I have called a database connection
    -----------------------------------------------
    $conn = intialitationDB();
    -----------------------------------------------
    $sql = "select * from tablebarang where merk = '".$merk."'";
    $result = mysqli_query($conn, $sql);
    $addNewProducts = array();
    foreach($result as $products){
        $tempSKU = searchEcommerceProducts($products['sku'], $shopID);

        if($tempSKU === false){
            $newProducts['sku'] = $products['sku'];
            $newProducts['catID'] = getCatIdEcommerece($merk, $products['sku']);
            $images = getPhotosSomeWebsite($merk, $products['sku']);
            for($i=1; $i <= 5; $i++){
                if(isset($images[$i-1])) $img = 'https://somewebsite.com/file/'.$images[$i-1];
                else $img = '';

                $newProducts['images_'.$i] = $img;
            }
            $addNewProducts['sku'][] = $products['sku'];
            $addNewProducts['item'][] = $newProducts;
        }
        // Markup
        // In here i should respone to user one by one, but i cannot because 
        i just need respone just some variable
    }
    return $addNewProducts;
    mysqli_close($conn);
}

function searchEcommerceProducts($sku, $shopID){
    $q = str_replace(" ","%2B", $sku);
    $url = "https://somewebsite.com/search/product?shop_id=$shopID&ob=11&rows=80&start=0&device=desktop&source=shop_product&q=$q";
    $html = file_get_contents($url);
    $html = json_decode($html, true);
    if($html["header"]["total_data"] >= 1) return true;
    else return false;
}

function getCatIdEcommerece($merk, $sku){
    $merk = str_replace(" ","%2B",$merk);
    $sku = str_replace(" ","%2B",$sku);
    $search = $merk . "%2B" . $sku;
    $url = "https://somesite.com/search/product/v3?scheme=https&device=desktop&catalog_rows=0&rows=1&source=search&ob=23&st=product&q=".$search."&sc=0&user_id=470833&rows=60&unique_id=35e66d287a5d4cefb1a674678be311f4";
    $html = file_get_contents($url);
    $html = json_decode($html, true);

    if (isset($html['data']['products'][0]['url'])){
        $url = $html['data']['products'][0]['url'];
        $cat_id = after_last ("catid%3D", $url);
    }else $cat_id = '';

    return $cat_id;
}

function getPhotosSomeWebsite($merk, $sku){
    $search = str_replace(' ','%20',$merk.' '.$sku);
    // assuming I have called a function name theCURL
    -----------------------------------------------
    $getFoto = theCURL("https://somesite.com/search_items/?by=relevancy&keyword=$search&limit=1&match_id=16775174&newest=0&order=desc&page_type=shop&__classic__=1",'GET','');
    -----------------------------------------------

    $getFoto = json_decode($getFoto, true);
    $items = $getFoto['items'];
    if(!empty($items)){
        $idProduct = $items[0]["itemid"];
        $getFoto = theCURL("https://somesite.com/item/get?itemid=$idProduct&shopid=16775174&__classic__=1",'GET','');
        $getFoto = json_decode($getFoto, true);
        return $getFoto['item']['images'];
    } else return null;
}

Explanation :

First : I Need get variable of SKU, CategoryID, ImagesURL from my scraping website where I have not input the item in my shop, so i dont need check double product item / duplicate product.

Second : I Call function getNewProductToEcommerce() from other class and just return variable SKU, CategoryID, ImagesURL in array. I need to save the product item to xlsx.

The Problem :

In looping i just want get some product where I have not input the item in my shop, but looping for 1000 sku product it taking too long time and return Request Timeout (just working for 100 sku product checking).

I have been duping the script using echo in looping foreach it's working fine for this 1000 sku product, but when i passing function it can't be and return error 'headers already sent by header'.

My Question

How i can get return function for many product without getting error 'headers already sent by header' or request time out in my problem ?

I am very grateful if someone can help me on this issue, thank you.

Answer 1

'Headers already sent' error occurs because PHP is printing some errors or warnings before you send the headers. Find what kind of error occurs. Most probably it is a php execution timeout issue in your case so try to increase the execution time limit in your php configuration file according to your needs:

In php.ini file:

max_execution_time=500

Also for memory limit:

memory_limit = 256M

If you don't have sufficient privileges to do that you may try to do your operations batch wise as a cron job.