Apache 2.2: Partial results are valid but processing is incomplete, unable to stat file (X-Sendfile)

Posted on June 9th, 2011 webmaster 5 comments          Print

Error Message:

(70008) Partial results are valid but processing is incomplete: xsendfile: unable to stat file: //server_name/folder/file

I deployed my code to our development box over the weekend which uses the X-Sendfile module for Apache to serve files but ran into this issue when I tried to download a file.  The X-Sendfile module worked fine on my local machine.  The only difference I could find was our development box was running Windows XP 64-bit and I’m running 32-bit (this is why it’s a good idea to try to have the machines in all your environments to be identical, VMs are great for this).

I couldn’t find a solution for my exact problem so I decided to check out the source code and searched for the error message “unable to stat file”:

  /* stat (for etag/cache/content-length stuff) */
  if ((rv = apr_file_info_get(&finfo, APR_FINFO_NORM, fd)) != APR_SUCCESS) {
    ap_log_rerror(
      APLOG_MARK,
      APLOG_ERR,
      rv,
      r,
      "xsendfile: unable to stat file: %s",
      translated
      );
    apr_file_close(fd);
    ap_remove_output_filter(f);
    ap_die(HTTP_FORBIDDEN, r);
    return HTTP_FORBIDDEN;
  }

It appears that the apr_file_info_get() function was not returning APR_SUCCESS on our development box for some reason.  I did some research on that function and found an old article that mentions something about one of the arguments may be beyond the OS file system support and would always return APR_INCOMPLETE (http://dev.ariel-networks.com/apr/apr-tutorial/html/apr-tutorial-5.html).

I put a quick fix/hack to make it work in our environment by simply checking for both APR_SUCCESS and APR_INCOMPLETE and only throw the error when the status is neither of those.

I simply changed:

if ((rv = apr_file_info_get(&finfo, APR_FINFO_NORM, fd)) != APR_SUCCESS)

to

if ((rv = apr_file_info_get(&finfo, APR_FINFO_NORM, fd)) != APR_SUCCESS	  && (rv = apr_file_info_get(&finfo, APR_FINFO_NORM, fd)) != APR_INCOMPLETE)

I then did a compile/build on Windows (this is another story), redeployed the module, and it finally worked!  I also checked the MD5 and SHA-1 hashes of a couple of files served through mod_xsendfile just to make sure it’s working properly and they matched the original hashes.

You can download the source and the .so file (if you trust me enough ) with the fix from here.

How to convert a Java keystore (JKS) to PEM format

Posted on May 8th, 2011 webmaster No comments          Print

I’m currently working on a new project where I need to write a service to serve static files to users.  Our web application currently runs on Tomcat alone as we don’t have many users (internal users only) and most of the content is dynamic.  But with this new project it just seems to make sense to put an Apache web server in front of Tomcat and use the mod_xsendfile module to serve the files, which will also allow us to control which users can access which files.

So I started playing with Apache, mod_xsendfile, and mod_proxy and finally got things working.  The last step is to add encryption.  The Java keystore format won’t work with Apache, however, so I needed a way to export the certificate and private key from the Java keystore we used for Tomcat and import it to a new PEM file so I can use it with Apache.

If we’re controlling the Certification Authority (CA) I probably would’ve just generated a new certificate, but this is unfortunately not the case in our environment (big company thing).  I would need to generate a certificate signing request, open a ticket with corporate IT, wait for approval, wait for someone to do it and send it to me, etc.  The process could take a few days so I decided to just do some Googling on how to extract the keys/certificates from the keystore and convert it to PEM which Apache web server will accept.

There doesn’t seem to be a quick way to directly convert from JKS to PEM so I had to convert from JKS to PKCS#12 first, then to PEM.

Here are the steps I took to do the conversion:

1.  Export certificate from the Java keystore and import it to a new PKCS#12 keystore format using the Java keytool (C:\Program Files\Java\jre6\bin\keytool.exe by default on Windows).

keytool -importkeystore -srckeystore myapp.jks -destkeystore myapp.p12 -srcalias myapp-dev -srcstoretype jks -deststoretype pkcs12

2. Convert the new PKCS#12 file (myapp.p12) to PEM using openssl (openssl.exe is in the bin directory of the Apache installation on Windows).

openssl pkcs12 -in myapp.p12 -out myapp.pem

If you’re running Apache on *nix, you’re all set! But if you’re running on Windows (I know, I know), you will need to remove the passphrase from the PEM file.

3. (Optional depending on enviroment) Create a version of the PEM file with the passphrase removed.

You may get this message when using the certificate in Apache running on Windows:

SSLPassPhraseDialog builtin is not supported on Win32.

The solution is to remove the password/passphrase from the PEM file, so let’s create a version of the PEM file without the passphrase.

openssl rsa -in myapp.pem -out myapp_nopassphrase.pem
openssl x509 -in myapp.pem >>myapp_nopassphrase.pem

Reference the myapp_newpassphrase.pem in your httpd.conf, start the Apache service, and you’re good to go!

Sources:

• http://stackoverflow.com/questions/652916/converting-a-java-keystore-into-pem-format
• http://www.madboa.com/geek/openssl/#key-removepass

How to import data from a CSV file into a PostgreSQL database

Posted on April 19th, 2011 webmaster No comments          Print

I was working on my personal project last week and needed to import some data in CSV format to a PostgreSQL database.  I wanted only certain fields to be imported so I had to specify this in the command.  I’m new to PostgreSQL so it took me quite a bit of Googling to figure out exactly how to do this.

Here’s how I did it on Ubuntu 10.04 LTS, ‘glucosetracker’ is the name of my database‘:

psql glucosetracker

\COPY glucose_glucose(owner_id, glucose_value, category_id, record_date, record_time, notes, created_on, last_updated_on) from '/home/jc/Downloads/jc_glucose_data.csv' DELIMITERS ',' CSV;

Note that ‘COPY is not exactly the same as ‘\COPY’. The COPY command runs under the PostgreSQL backend user (default ‘postgres’) so this user will need read access to your CSV file.  The \COPY command runs under the user executing the command.  I was getting permission denied errors my first few tries because I was using ‘COPY’ instead of ‘\COPY’ since I thought they were they same thing.