This is for people who are considering purchasing real estate from Shelter Rock Builders. Shelter Rock is based in Great Neck, NY and is run by the brothers, Ben and Joe Shavolian though I've only had dealings with Joe. After I moved into my unit at Plaza 21, I noticed that the bathtub had been installed backwards, with the drain and shower body on opposite sides, making it impossible to take a bath comfortably because one's head would hit the faucet when lying in the tub in the correct orientation (the tub is designed to be lied in in a certain direction).

Despite repeated attempts to have it fixed (and hearing Joe spin different stories about why it couldn't be fixed with a new story each week), they have not addressed my issue after more than 6 months and have refused to show me proof of why it cannot be fixed.

Note also that the way that Shelter Rock writes its offering plans, they are not liable for anything after the closing. The condo board is. They are wiping themselves completely clean of any responsibility post-closing. So if you want to file a lawsuit post-closing due to a latent flaw, and you're suing the condo board, which you've bought into at closing.

[Update 12/6/11]
Joe came to my apartment a couple of weeks ago to look at the tub, and after calling me some not so nice names (have video proof), agreed to have the shower body moved to the opposite side of the tub. Not ideal to me, as it's obviously not how the bathroom was designed, but it seems that I have no choice as apparently, a beam was installed underneath the tub where there isn't supposed to be. The repair work is supposed to occur in mid-January. Will post another update once that has occurred.

[Update 2/23/11]
The repair work has been done and I'm satisfied with the work that they did. As my attorney said, I may have to swallow a little, but it's not a lot. Lessons learned.

Want to see which keywords a competitor is running on, and what ad text they’re using? If you give us a list of keywords and an example of one of their ads, we can run it through our tool to let you know which keywords on your list they are running on. We will also tell you ad text they are using, and what destination URL they are driving to. This is particularly useful if you have a long list of keywords, but don’t want to type each one in to see if the competitor is showing. For example, let’s say that my competitor is a particular advertiser that is using NovoMedLink.com as their display URL, and then I suspect that they may be targeting some of the following keywords.

• diabetes
• diabetes treatment
• type 2 diabetes
• type 2 diabetes treatment
• treating diabetes
• treatment for type 2 diabetes
• a1c
• glucagon
• glycemic control
• insulin

Then we will deliver an excel report that lists the keyword, whether a match was found, the ad text and the final destination URL after any redirects. See screenshot below. Excel version here.
If you are interested in this, please contact us and we can provide a price quote.

Boatdig.com is a sailboat classifieds website that we just launched. It has been designed to allow ads related to certain boat models to be grouped together, geo mapping, and optimized to allow for maximum visibility of individual ads both from a search engine perspective as well as from a site perspective.

My web hosting adventures have spanned several different hosts over the years. I started off with Lunarpages - I did my research and found that they were consistently touted in the reviews that I had read. At the time (this is no longer the case), with a shared hosting account on Lunarpages you could host only one domain without paying extra money. As I started wanting things like the ability to quickly and easily host additional domains, get a dedicated IP addresses and SSH access, it seemed that Lunarpages no longer met my needs. So then I switched to Godaddy and got a virtual dedicated server (VDS or VPS) with the Plesk control panel. With this, I felt like I had more control. However, as I learned more about web hosting, I started to see how the Plesk control panel was getting in the way for me. There was a limitation in how many domains I could add with my plan (if I wanted to add more, I’d have to shell out more money) and I got worried that if I upgraded certain packages on the VDS, that it would break Plesk. So I started looking into VDSes without a control panel. I had installed a linux distribution on my local machine and thought that I would be able to handle the administration of a web server from the command line. That’s when I found Slicehost through a recommendation in an IRC channel. Slicehost offers VDSes with dedicated resources (memory and CPU allocation) and you get to choose your linux distribution. Starting at just $20, this is a really good deal in my opinion. I had full control of my VDS including which packages and versions I wanted to install and I could also host as many domains as I wanted by using virtual hosting on either Apache or Lighttpd (I tried both). However, all this control came at a sharp price to me. I had to spend a lot of time learning linux commands, managing packages and figuring out optimal configurations.

I was still fairly happy with Slicehost and I wasn’t planning on switching hosts again. I had a couple of Ruby on Rails applications that I wanted to launch and I was having a really hard time with deployment. I figured that I could find an inexpensive managed server host for just my Rails apps. I did a search in Google and found Hosting Rails. I asked about them in the #rubyonrails IRC channel and heard positive things about them, so I went ahead and signed up for a plan.

It turns out that I didn’t realize all the features I was getting until I actually started using it (yes, I know that they’re all laid out on the home page). With hosting plans starting at $2.88 per month, you get all the expected programs – cPanel, php, mysql, perl etch. In addition to these, you also get SSH access, subversion and the ability to host unlimited domains – I’ve never heard of getting all this on a shared hosting plan. Some people may say that you don’t really need SSH access if you have a cPanel. Well, the truth is if you want to move files around, edit them and create symlinks, it’s a lot faster if you have SSH access.

So now, I’m spending less money on hosting that I did with a VDS, I’m able to host all my sites just as I did before and I’m spending less time doing systems administration because I don’t have to worry about maintaining and updating software packages because the web host does it for me.

If you plan on signing up for an account, please consider using my referral link.

We just launched a new web application called Telecommuter Jobs. A Ruby on Rails application, it scrapes job listings from Craigslist that have been labeled as being appropriate for telecommuting. It is specific to job category but not to location, which makes sense since telecommuting jobs should be able to be done from anywhere. Listings automatically update once a day.

Click for largerI've been seeing a lot of ads for Microsoft adcenter lately. After seeing this (on a celebrity gossip blog), I've concluded that I'm being shown so many of them because of behavioral targeting. Surely, they don't think that visitors of celebrity gossip blogs have a high composition of Internet advertisers. However, it's my opinion that some frequency capping might be in order. I already have an adcenter account, so the 50-100 impressions I've seen are wasted.

I've recently been messing around with Jens Kraemer's fantastic web crawler module, Rdig. One thing I noticed about it, however is that it's not coded to optimally crawl a website while including or excluding certain URL patterns.

First off, the start_url has to get past the include and exclude url pattern filters or else the site won't get crawled.

Secondly, because documents go through the pattern filter before they get added to the queue, pages that can be accessed only from pages that don't get past the pattern filter won't be seen at all.

The solution to this is fairly simple. I altered the code so that documents would go through the filters except for the include and exclude filters before it gets added to the queue, and then run the documents through the include and exclude filters before it gets added to the index. So here is what I did:

Changed the order of members of the filter_chain array in rdig.rb:

    def filter_chain
      @filter_chain ||= {
        # filter chain for http crawling
        :http => [
          :scheme_filter_http,
          :fix_relative_uri,
          :normalize_uri,
          { :hostname_filter => :include_hosts },
          RDig::UrlFilters::VisitedUrlFilter,         
          { RDig::UrlFilters::UrlInclusionFilter => :include_documents },
          { RDig::UrlFilters::UrlExclusionFilter => :exclude_documents } 
        ],
        # filter chain for file system crawling
        :file => [
          :scheme_filter_file,
          { RDig::UrlFilters::PathInclusionFilter => :include_documents },
          { RDig::UrlFilters::PathExclusionFilter => :exclude_documents }
        ]
      }
         
    end

Replaced the definition of the apply (line 55) method in url_filters.rb with the following:

      def apply_first(document) # applies 0-4 of @filters array
        @filters[0..4].each { |filter|
          return nil unless filter.call(document)
        }
        return document
      end
      
      def apply_second(document) # applies 5-6 of @filters array
        @filters[5..6].each { |filter|
          return nil unless filter.call(document)
        }
        return document
      end

In the add_url method definition in crawler.rb, I changed apply method call to the following:

      doc = filterchain.apply_first(doc)

Changed the process_document method definition in crawler.rb to the following:

    def process_document(doc, filterchain)
      doc.fetch
      # add links from this document to the queue
      doc.content[:links].each { |url| 
        add_url(url, filterchain, doc) 
      } unless doc.content[:links].nil?

      return unless @etag_filter.apply(doc)
      doc = filterchain.apply_second(doc)
      if doc
        @indexer << doc if doc.needs_indexing?
      end
    rescue
      puts "error processing document #{doc.uri.to_s}: #{$!}"
      puts "Trace: #{$!.backtrace.join("\n")}" if RDig::config.verbose
    end

Also, if you're having problems appending to an existing index, make sure that line 103 of config.rb starts with 'cfg' and not 'config'.

updated Oct 23, 2007

I recently decided to switch from Apache to lighttpd because Apache along with mod_php was using up a lot of memory and Lighttpd apparently has a lighter memory footprint. The difference is significant - with Apache, I was using up nearly all of my 256MB whereas with Lighttpd, I'm using only 132MB and I think with some tuning, I'll be able to get it lower.

In order to install lighttpd with yum, you need to include the RPMForge repository. Instructions on how to do this are here.

For some reason the RPMForge repository only showed version 1.3.16 for me, so I used RPMs, which can be grabbed here. Make sure that you grab the one that matches your distro and system architecture. If you use php, you'll want to install the fastcgi rpm as well. To install, I did:

$ wget http://dag.wieers.com/rpm/packages/lighttpd/lighttpd-1.4.18-1.el4.rf.x86_64.rpm
$ wget http://dag.wieers.com/rpm/packages/lighttpd/lighttpd-fastcgi-1.4.18-1.el4.rf.x86_64.rpm
$ rpm -Uvh lighttpd-1.4.18-1.el4.rf.x86_64.rpm
$ rpm -Uvh lighttpd-fastcgi-1.4.18-1.el4.rf.x86_64.rpm

You should then be able to start up the daemon with:

$ service lighttpd start

The configuration file is /etc/lighttpd/lighttpd.conf. In it, you should set your web root with the server.document-root directive.

Getting lighttpd to serve PHP files

Make sure that you installed lighttpd-fastcgi above. Make sure that the following line exists in /etc/php.ini though it should have the right setting by default:

cgi.fix-pathinfo = 1

Make sure mod_fastcgi is uncommented (and thus loaded) in server.modules in your lighttpd.conf file:

Server.modules = (
	mod_fastcgi,
			)

Then make sure the fast.server section is uncommented and looks like as follows:

fastcgi.server = ( ".php" => (( 
                     "bin-path" => "/path/to/php-cgi",
                     "socket" => "/tmp/php.socket"
                 )))

My bin-path was /usr/bin/php-cgi. If you're unsure, you can issue the following command:

$ whereis php-cgi

Then restart the lighttpd daemon with:

$ service lighttpd restart

And you should be able to serve php files.

Virtual Hosts

Virtual hosting can be done in a few different ways: using conditionals, simple-vhosts or a combination of conditionals and simple-vhosts. As I understand it, simple-vhosts are used for where all your virtual hosts (domains, subdomains or sites) have the same structure within the web root and conditionals are used for exceptions.

Since I have a relatively small number of sites, I decided to go the route of using conditionals. For documentation on this, take a look at this wiki article.

In lighttpd.conf, I used the following as an example:

$HTTP[host] == domain.com {
server.document-root = /path/to/document/root/for/domain.com
}
$HTTP[host] == domain2.com {
server.document-root = /path/to/document/root/for/domain2.com
}

Using a combination of conditionals and simple-vhosts can save you some typing. For example:

$HTTP["host"] == "subdomain1.example.org" {
 #conditional
}
else $HTTP["host"] == "subdomain2.example.org" {
 #conditional
}
else $HTTP["host"] =~ "^." {
 # simple vhost stuff here
}

Redirecting domain.com to www.domain.com

If you'd like to redirect requests made to domain.com to www.domain.com you can't do it using an .htaccess file because lighttpd doesn't support them. You can, however enable mod_redirect in lighttpd.conf (uncomment it at the top of the file) and use something like the following:

$HTTP["host"] !~ "^(www|mail|webmail|208)" {
  $HTTP["host"] =~ "^(.*)" {
    url.redirect = ("^/(.*)" => "http://www.%1/$1")
  }
}

What that does is redirects any http requests that do not start with www, mail, webmail or 208 to the same url except beginning with "http://www." I have "208" in there as well because if I type in my server IP address, I don't want it to be redirected.

You can also include this within a virtual host so that this conditional affects only certain domains.

Drupal Clean URLs

Using mod_rewrite seemed to be the easiest way to enable clean URLs in drupal. You will want to enable mod_rewrite (uncomment it out at the top of your lighttpd.conf) and then use the following code:

url.rewrite-final = (
  "/rss.xml$" => "/index.php?q=rss.xml",
  "^/([^.?]*)\?(.*)$" => "/index.php?q=$1&$2",
  "^/search/(.*)$" => "/index.php?q=search/$1",
  "^/([^.?]*)$" => "/index.php?q=$1",
  "^/([^.?]*\.html)$" => "/index.php?q=$1",
  "^/([^.?]*\.htm)$" => "/index.php?q=$1"
)

I have multiple sites on my server and only a portion of them are drupal sites, so I put those rewrite rules into a vhost:

$HTTP["host"] == "www.mydomain.com" {
  server.document-root = "/var/www/html/drupal"
  url.rewrite-final = (
    "/rss.xml$" => "/index.php?q=rss.xml",
    "^/([^.?]*)\?(.*)$" => "/index.php?q=$1&$2",
    "^/search/(.*)$" => "/index.php?q=search/$1",
    "^/([^.?]*)$" => "/index.php?q=$1",
    "^/([^.?]*\.html)$" => "/index.php?q=$1",
    "^/([^.?]*\.htm)$" => "/index.php?q=$1"
  )
}

You may be puzzled by why the server.document-root is set to something that looks rather generic, but that is because I use drupal's multisite feature, so all my drupal sites use that same document root.

Wordpress permalinks

Wordpress permalinks can be enabled in a very similar way as with drupal. Mod_rewrite should be enabled, and then the following code used:

url.rewrite-final = (
  "^/(wp-.+).*/?" => "$0",
  "^/(sitemap.xml)" => "$0",
  "^/(xmlrpc.php)" => "$0",
  "^/(.+)/?$" => "/index.php/$1"
)

And for a specific virtual host:

$HTTP["host"] == "www.wordpress_site.com" {
  server.document-root = "/var/www/html/domains/wordpress_site"
  url.rewrite-final = (
    "^/(wp-.+).*/?" => "$0",
    "^/(sitemap.xml)" => "$0",
    "^/(xmlrpc.php)" => "$0",
    "^/(.+)/?$" => "/index.php/$1"
  )
}