Supermind Search Consulting Blog 
Solr - ElasticSearch - Big Data

Posts about PHP

Send response to client in PHP and continue processing

Posted by Kelvin on 03 Feb 2014 | Tagged as: PHP

Here's one way to send and close the connection to the client and for the PHP script to continue processing, presumably to perform some processing that is time-consuming:

span class="st0">"Connection: close\r\n""Content-Encoding: none\r\n"// optional
'Text user will see'"Content-Length: $size"// Strange behaviour, will not work
// Unless both are called !
//do processing here
'Text user will never see');
//do some processing
 

Note that some stackoverflow answers which mention the use of ignore_user_abort are mistaken. That's not required at all. And you'll need the Content-Encoding: none header, otherwise it won't work properly with clients that accept gzip encoding for example.

Interesting PHP and apache/nginx links

Posted by Kelvin on 25 Nov 2012 | Tagged as: PHP, programming

http://code.google.com/p/rolling-curl/
A more efficient implementation of curl_multi()

https://github.com/krakjoe/pthreads
http://docs.php.net/manual/en/book.pthreads.php
Posix threads in PHP. Whoa!

http://www.underhanded.org/blog/2010/05/05
Installing Apache Worker over prefork.

http://www.wikivs.com/wiki/Apache_vs_nginx
I stumbled on this page when researching the pros/cons of Apache + mod_php vs nginx + php5-fpm

http://barry.wordpress.com/2008/04/28/load-balancer-update/
Nice posting about wordpress.com's use of nginx for load balancing.

Download KhanAcademy videos with a PHP crawler

Posted by Kelvin on 08 Oct 2011 | Tagged as: PHP, programming

At the moment (October 2011), there's no simple way to download all videos from a playlist from KhanAcademy.org.

This simple PHP crawler script changes that. :-)

What it does is downloads the videos (from archive.org) to a subfolder, numbering and naming the videos with the respective titles (not the gibberish titles that archive.org has assigned them). Additionally, through the use of wget –continue, the crawler has auto-resume support, so even if your computer crashes in the middle of a crawl, you don't need to start all over again.

Usage

Usage is like this, assuming the script is named downkhan.php:

 

where folder is the subdirectory to save the videos in, and urls.txt is a list of urls obtained by running a regex on http://www.khanacademy.org/#browse.

Regex

The regex used was

href="(.*?)".*?><span.*?>(.*?)</span>
 

urls

Here is a few lines of a urls.txt file:

 

Here's a list of what I've created so far:

http://www.supermind.org/code/history.txt
http://www.supermind.org/code/biology.txt
http://www.supermind.org/code/finance.txt
http://www.supermind.org/code/cosmology.txt
http://www.supermind.org/code/healthcare.txt
http://www.supermind.org/code/linearalgebra.txt
http://www.supermind.org/code/statistics.txt

script code

And here's the script:

span class="st0">'argv'"\n""/""|""Fetching $url\n";
  $html = ""/<a href=\"(http:\/\/www.archive.org.*?)\"/""$counter. $title.mp4""$outfile"// initialize curl handle
// set url to post to
// allow redirects
// return into a variable
'Accept: */*', 'User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows)'// run the whole process
'Curl error: '/* return if already absolute URL */"/* queries and anchors */'#' || $rel[0] == '?'/* parse base URL and convert to local variables:
 $scheme, $host, $path */
/* remove non-directory element from path */'#/[^/]*$#', ", $path);

    /* destroy path if relative url points to root */'/') $path = ";

    /* dirty absolute URL */"$host$path/$rel";

    /* replace '//' or '/./' or '/foo/../' with '/' */'#(/\.?/)#', '#/(?!\.\.)[^/]+/\.\./#''/'/* absolute URL is ready! */'://'

Painless CRUD in PHP via AjaxCrud

Posted by Kelvin on 08 Oct 2011 | Tagged as: PHP, programming

I recently discovered an Ajax CRUD library which makes CRUD operations positively painless: AjaxCRUD

Its features include:

– displaying list in an inline-editable table
– generates a create form
– all operations (add,edit,delete) handled via ajax
– supports 1:many relations
– only 1 class to include!!

I highly recommend you try it out!

Here is the example code:

# the code for the class
'ajaxCRUD.class.php');

# this one line of code is how you implement the class
"Customer",
                             "tblCustomer", "pkCustomerID");

# don't show the primary key in the table
# my db fields all have prefixes;
# display headers as reasonable titles
"fldFName", "First""fldLName", "Last""fldPaysBy", "Pays By""fldDescription", "Customer Info");

# set the height for my textarea
$tblCustomer->setTextareaHeight('fldDescription', 100);

# define allowable fields for my dropdown fields
# (this can also be done for a pk/fk relationship)
"Cash", "Credit Card", "Paypal""fldPaysBy", $values);

# add the filter box (above the table)
$tblCustomer->addAjaxFilterBox("fldFName");

# actually show to the table
$tblCustomer->showTable();
 

PHP function to send an email with file attachment

Posted by Kelvin on 11 Jun 2011 | Tagged as: PHP, programming

Courtesy of http://www.finalwebsites.com/forums/topic/php-e-mail-attachment-script

span class="st0">"r""From: "" <"">\r\n""Reply-To: ".$replyto."\r\n""MIME-Version: 1.0\r\n""Content-Type: multipart/mixed; boundary=\"".$uid."\"\r\n\r\n""This is a multi-part message in MIME format.\r\n""–".$uid."\r\n""Content-type:text/plain; charset=iso-8859-1\r\n""Content-Transfer-Encoding: 7bit\r\n\r\n""\r\n\r\n""–".$uid."\r\n""Content-Type: application/octet-stream; name=\"""\"\r\n"; // use different content types here
"Content-Transfer-Encoding: base64\r\n""Content-Disposition: attachment; filename=\"""\"\r\n\r\n""\r\n\r\n""–".$uid."–""""mail send … OK"; // or use booleans here
"mail send … ERROR!";
    }
}
 

Prettyprint xml in PHP

Posted by Kelvin on 04 Dec 2010 | Tagged as: PHP

Ever wanted to format your XML nicely? Use the SimpleDOM class.

Usage is like so:

span class="st0">"SimpleDOM.php";

$xml = "<foo><bar>car</bar></foo>"

Produces:

<?xml version="1.0"?>
<foo>
  <bar>car</bar>
</foo>
 

URLizer: a WordPress plugin to automatically linkify URLs

Posted by Kelvin on 12 Oct 2010 | Tagged as: PHP, programming

Am I the only guy using WordPress who is too lazy to type out anchors?

Well, I've been using a WordPress plugin I wrote to automagically linkify URLs for a number of years now, and finally decided to add it to Google Code.

So here it is! http://code.google.com/p/urlizer/

Run php from html files on Dreamhost

Posted by Kelvin on 10 Oct 2010 | Tagged as: PHP, programming

Modify .htaccess to include this:

Correct

 

WRONG

 

or

 

[SOLVED] Howto build the PHP rrdtool extension

Posted by Kelvin on 09 Oct 2010 | Tagged as: PHP, programming, Ubuntu

The definitive answer is here: http://www.samtseng.liho.tw/~samtz/blog/2009/03/11/howto-build-the-php-rrdtool-extension/

If you're on Ubuntu, do this first:

 

Then follow the steps above.

[SOLVED] curl: (56) Received problem 2 in the chunky parser

Posted by Kelvin on 09 Oct 2010 | Tagged as: crawling, PHP, programming

The problem is described here:

http://curl.haxx.se/mail/lib-2006-04/0046.html

I successfully tracked the problem to the "Connection:" header. It seems that
if the "Connection: keep-alive" request header is not sent the server will
respond with data which is not chunked . It will still reply with a
"Transfer-Encoding: chunked" response header though.
I don't think this behavior is normal and it is not a cURL problem. I'll
consider the case closed but if somebody wants to make something about it I
can send additional info and test it further.

The workaround is simple: have curl use HTTP version 1.0 instead of 1.1.

In PHP, add this:

 

Next Page »