Home Page
Back



Written:
25-Jan-2001

Ad and Counter Blocking
By Thiravudh Khoman

Here's something I recently implemented on my Windows machine that helps speed up web browsing a bit. (Note: This NOT an original idea, not by a long shot.)

When you give your web browser a URL such as https://www.yahoo.com, the usual way this name is "resolved" or converted into a numerical IP address is by the use of DNS servers. On smaller networks that run TCP/IP (such as on a local area network), it's not necessary to use a full-blown DNS server to do name resolution. Instead, you can compile a list of IP addresses and domain names and put them into a file called HOSTS. All TCP/IP-capable operating systems, regardless of whether they're running DOS, Windows, Mac, Linux, or whatever CAN make use of this HOSTS file.

On Windows machines, this file is usually located in C:\Windows, while on other O/S'es it's located elsewhere. (If I remember correctly, with Linux it's in /etc. Not sure where it's located on a Mac, but probably in one of the system folders.) Chances are, though, on most Windows machines you won't find an existing HOSTS file; instead, you'll find a HOSTS.SAM file which is merely a "sample" HOSTS file. You can copy HOSTS.SAM to HOSTS to get a barebones start.

The format of the HOSTS file is very simple. Each line contains an IP address, some white space (usually a tab), and then a domain name associated with that IP addresss. In most HOSTS files, you will also see a rather odd entry: 127.0.0.1 localhost. "localhost" is a generic name given to YOUR computer and 127.0.0.1 is the dummy IP address assigned to this name. In actual fact, ALL computers on the internet have this entry, and because 127.0.0.1 is one of those magic non routable IP addresses, it will never exist on the real internet, thus avoiding any possibility of address conflicts.

More often than not, localhost is used for "loopback" testing; e.g. to see if your TCP/IP is working properly. Try it. Type: ping localhost. On my machine, this is the output I get.

    Pinging thirav.eug.or.home.com [127.0.0.1] with 32 bytes of data:

    Reply from 127.0.0.1: bytes=32 time<10ms TTL=128
    Reply from 127.0.0.1: bytes=32 time<10ms TTL=128
    Reply from 127.0.0.1: bytes=32 time<10ms TTL=128
    Reply from 127.0.0.1: bytes=32 time<10ms TTL=128

    Ping statistics for 127.0.0.1:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
    Approximate round trip times in milli-seconds:
    Minimum = 0ms, Maximum = 0ms, Average = 0ms

With no lost packets, my computer is deemed able to communicate with "localhost", as it very well should, since I'm just ping'ing my own computer.

Now, if you have a functioning HOSTS file, TCP/IP can use BOTH the HOSTS file AND your duly assigned DNS servers, which is what I've done. To be precise, I've entered the names of some "undesirable servers" and assigned them the magic IP number 127.0.0.1, meaning from the standpoint of MY computer, those domain names now exist locally (although their content doesn't). Needless to say, this doesn't affect any other computer on the internet.

At present, this is what my HOSTS file looks like:

    127.0.0.1    localhost
    127.0.0.1    ads.mweb.co.th
    127.0.0.1    ads2.mweb.co.th
    127.0.0.1    usa.nedstat.net
    
    127.0.0.1    pop3.norton.antivirus
    127.0.0.1    pop3.spa.norton.antivirus
    
    127.0.0.1    207-87-18-203.wsmg.digex.net
    127.0.0.1    Garden.ngadcenter.net
    127.0.0.1    Ogilvy.ngadcenter.net
    127.0.0.1    ResponseMedia-ad.flycast.com
    127.0.0.1    Suissa-ad.flycast.com
    127.0.0.1    UGO.eu-adcenter.net
    127.0.0.1    VNU.eu-adcenter.net
    127.0.0.1    a32.g.a.yimg.com
    127.0.0.1    ad-adex3.flycast.com
    127.0.0.1    ad.adsmart.net
    127.0.0.1    ad.ca.doubleclick.net
    127.0.0.1    ad.de.doubleclick.net
    127.0.0.1    ad.doubleclick.net
    127.0.0.1    ad.fr.doubleclick.net
    127.0.0.1    ad.jp.doubleclick.net
    127.0.0.1    ad.linkexchange.com
    127.0.0.1    ad.linksynergy.com
    127.0.0.1    ad.nl.doubleclick.net
    127.0.0.1    ad.no.doubleclick.net
    127.0.0.1    ad.preferences.com
    127.0.0.1    ad.sma.punto.net
    127.0.0.1    ad.uk.doubleclick.net
    127.0.0.1    ad.webprovider.com
    127.0.0.1    ad08.focalink.com
    127.0.0.1    adcontroller.unicast.com
    127.0.0.1    adcreatives.imaginemedia.com
    127.0.0.1    adex3.flycast.com
    127.0.0.1    adforce.ads.imgis.com
    127.0.0.1    adforce.imgis.com
    127.0.0.1    adfu.blockstackers.com
    127.0.0.1    adimage.blm.net
    127.0.0.1    adimages.earthweb.com
    127.0.0.1    adimg.egroups.com
    127.0.0.1    admedia.xoom.com
    127.0.0.1    adpick.switchboard.com
    127.0.0.1    adremote.pathfinder.com
    127.0.0.1    ads.admaximize.com
    127.0.0.1    ads.bfast.com
    127.0.0.1    ads.clickhouse.com
    127.0.0.1    ads.enliven.com
    127.0.0.1    ads.fairfax.com.au
    127.0.0.1    ads.fool.com
    127.0.0.1    ads.freshmeat.net
    127.0.0.1    ads.hollywood.com
    127.0.0.1    ads.i33.com
    127.0.0.1    ads.infi.net
    127.0.0.1    ads.jwtt3.com
    127.0.0.1    ads.link4ads.com
    127.0.0.1    ads.lycos.com
    127.0.0.1    ads.madison.com
    127.0.0.1    ads.mediaodyssey.com
    127.0.0.1    ads.msn.com
    127.0.0.1    ads.ninemsn.com.au
    127.0.0.1    ads.seattletimes.com
    127.0.0.1    ads.smartclicks.com
    127.0.0.1    ads.smartclicks.net
    127.0.0.1    ads.sptimes.com
    127.0.0.1    ads.tripod.com
    127.0.0.1    ads.web.aol.com
    127.0.0.1    ads.x10.com
    127.0.0.1    ads.xtra.co.nz
    127.0.0.1    ads.zdnet.com
    127.0.0.1    ads01.focalink.com
    127.0.0.1    ads02.focalink.com
    127.0.0.1    ads03.focalink.com
    127.0.0.1    ads04.focalink.com
    127.0.0.1    ads05.focalink.com
    127.0.0.1    ads06.focalink.com
    127.0.0.1    ads08.focalink.com
    127.0.0.1    ads09.focalink.com
    127.0.0.1    ads1.activeagent.at
    127.0.0.1    ads10.focalink.com
    127.0.0.1    ads11.focalink.com
    127.0.0.1    ads12.focalink.com
    127.0.0.1    ads14.focalink.com
    127.0.0.1    ads16.focalink.com
    127.0.0.1    ads17.focalink.com
    127.0.0.1    ads18.focalink.com
    127.0.0.1    ads19.focalink.com
    127.0.0.1    ads2.zdnet.com
    127.0.0.1    ads20.focalink.com
    127.0.0.1    ads21.focalink.com
    127.0.0.1    ads22.focalink.com
    127.0.0.1    ads23.focalink.com
    127.0.0.1    ads24.focalink.com
    127.0.0.1    ads25.focalink.com
    127.0.0.1    ads3.zdnet.com
    127.0.0.1    ads3.zdnet.com
    127.0.0.1    ads5.gamecity.net
    127.0.0.1    adserv.iafrica.com
    127.0.0.1    adserv.quality-channel.de
    127.0.0.1    adserver.dbusiness.com
    127.0.0.1    adserver.garden.com
    127.0.0.1    adserver.janes.com
    127.0.0.1    adserver.merc.com
    127.0.0.1    adserver.monster.com
    127.0.0.1    adserver.track-star.com
    127.0.0.1    adserver1.ogilvy-interactive.de
    127.0.0.1    adtegrity.spinbox.net
    127.0.0.1    antfarm-ad.flycast.com
    127.0.0.1    au.ads.link4ads.com
    127.0.0.1    banner.media-system.de
    127.0.0.1    banner.orb.net
    127.0.0.1    banner.relcom.ru
    127.0.0.1    banners.easydns.com
    127.0.0.1    banners.looksmart.com
    127.0.0.1    banners.wunderground.com
    127.0.0.1    barnesandnoble.bfast.com
    127.0.0.1    beseenad.looksmart.com
    127.0.0.1    bizad.nikkeibp.co.jp
    127.0.0.1    bn.bfast.com
    127.0.0.1    c3.xxxcounter.com
    127.0.0.1    califia.imaginemedia.com
    127.0.0.1    cds.mediaplex.com
    127.0.0.1    click.avenuea.com
    127.0.0.1    click.go2net.com
    127.0.0.1    click.linksynergy.com
    127.0.0.1    cookies.cmpnet.com
    127.0.0.1    cornflakes.pathfinder.com
    127.0.0.1    counter.hitbox.com
    127.0.0.1    crux.songline.com
    127.0.0.1    erie.smartage.com
    127.0.0.1    etad.telegraph.co.uk
    127.0.0.1    fp.valueclick.com
    127.0.0.1    gadgeteer.pdamart.com
    127.0.0.1    gm.preferences.com
    127.0.0.1    gp.dejanews.com
    127.0.0.1    hg1.hitbox.com
    127.0.0.1    image.click2net.com
    127.0.0.1    image.eimg.com
    127.0.0.1    images2.nytimes.com
    127.0.0.1    jobkeys.ngadcenter.net
    127.0.0.1    kansas.valueclick.com
    127.0.0.1    leader.linkexchange.com
    127.0.0.1    liquidad.narrowcastmedia.com
    127.0.0.1    ln.doubleclick.net
    127.0.0.1    m.doubleclick.net
    127.0.0.1    macaddictads.snv.futurenet.com
    127.0.0.1    maximumpcads.imaginemedia.com
    127.0.0.1    media.preferences.com
    127.0.0.1    mercury.rmuk.co.uk
    127.0.0.1    mojofarm.sjc.mediaplex.com
    127.0.0.1    nbc.adbureau.net
    127.0.0.1    newads.cmpnet.com
    127.0.0.1    ng3.ads.warnerbros.com
    127.0.0.1    ngads.smartage.com
    127.0.0.1    nsads.hotwired.com
    127.0.0.1    ntbanner.digitalriver.com
    127.0.0.1    ph-ad05.focalink.com
    127.0.0.1    ph-ad07.focalink.com
    127.0.0.1    ph-ad16.focalink.com
    127.0.0.1    ph-ad17.focalink.com
    127.0.0.1    ph-ad18.focalink.com
    127.0.0.1    rd.yahoo.com
    127.0.0.1    realads.realmedia.com
    127.0.0.1    redherring.ngadcenter.net
    127.0.0.1    redirect.click2net.com
    127.0.0.1    regio.adlink.de
    127.0.0.1    retaildirect.realmedia.com
    127.0.0.1    s2.focalink.com
    127.0.0.1    sh4sure-images.adbureau.net
    127.0.0.1    spin.spinbox.net
    127.0.0.1    static.admaximize.com
    127.0.0.1    stats.superstats.com
    127.0.0.1    sview.avenuea.com
    127.0.0.1    thinknyc.eu-adcenter.net
    127.0.0.1    tracker.clicktrade.com
    127.0.0.1    tsms-ad.tsms.com
    127.0.0.1    v0.extreme-dm.com
    127.0.0.1    v1.extreme-dm.com
    127.0.0.1    van.ads.link4ads.com
    127.0.0.1    view.accendo.com
    127.0.0.1    view.avenuea.com
    127.0.0.1    w113.hitbox.com
    127.0.0.1    w25.hitbox.com
    127.0.0.1    web2.deja.com
    127.0.0.1    webads.bizservers.com
    127.0.0.1    www.PostMasterBannerNet.com
    127.0.0.1    www.ad-up.com
    127.0.0.1    www.admex.com
    127.0.0.1    www.alladvantage.com
    127.0.0.1    www.burstnet.com
    127.0.0.1    www.commission-junction.com
    127.0.0.1    www.eads.com
    127.0.0.1    www.freestats.com
    127.0.0.1    www.imaginemedia.com
    127.0.0.1    www.netdirect.nl
    127.0.0.1    www.oneandonlynetwork.com
    127.0.0.1    www.targetshop.com
    127.0.0.1    www.teknosurf2.com
    127.0.0.1    www.teknosurf3.com
    127.0.0.1    www.valueclick.com
    127.0.0.1    www.websitefinancing.com
    127.0.0.1    www2.burstnet.com
    127.0.0.1    www4.trix.net
    127.0.0.1    www80.valueclick.com
    127.0.0.1    z.extreme-dm.com
    127.0.0.1    z0.extreme-dm.com
    127.0.0.1    z1.extreme-dm.com
    

My HOSTS file is actually broken down into 3 sections. The first section (containing 4 lines at present), aside from the localhost entry, contains the sites of undesirable sites that *I* have compiled. The next section, set off by a blank line, contains 2 lines inserted by Norton Anti-Virus (NAV) that's used for email virus scanning. If you don't use NAV, leave this out. The last section is a list of undesirables obtained from https://www.ecst.csuchico.edu/~atman/spam/adblock.shtml, which is the source of this how-to. I highly suggest that you take a trip there and read the article.

Now, EXACTLY what are these sites? Basically, they fall into two categories:

  • The first type of undesirable site are those that serve up advertising. For example, ads.mweb.co.th and ads2.mweb.co.th are ad sites that sanook.com uses. Having assigned these sites the IP address 127.0.0.1 on my computer, whenever I go to www.sanook.com, sanook.com's homepage will now attempt to load these ads from 127.0.0.1 (i.e. on my computer) rather than from the real sites. Of course, the desired content doesn't exist on my computer and it will eventually "timeout". But since everything is done locally, the timeout should occur pretty quickly (presumably/hopefully faster than it would take to load the real ad).

    You can try this for yourself. Go to https://www.sanook.com. At the top of the page you will see a dark red rectangle with curved sides. If you have a HOSTS file with no references to ads.mweb.co.th or ads2.mweb.co.th, there will be an ad in the red area. On the other hand, if you were to use my HOSTS file, there will now be no ad, although the red area will still display. Not having to access the true mweb website should speed up your display a bit. It also removes one potential eyesore. To further test the case of the disappearing ad, you can also try https://www.dilbert.com and https://www.freshmeat.net.

  • The second type of undesirable site are those that count or "audit" your visit to a particular site. For example, the Bangkok Post's website (https://www.bangkokpost.com) uses a service by spinbox.net to obtain statistics on the number of visits to their website (or "eyeballs" in e-business vernacular). Presumably, this information is collected so that they can justify their online advertising rates or something. Now, if you don't want spinbox.net to count your visit or possibly worse, to obtain "other" information about you - e.g. your IP address (which determines which country you're in), where you just came from (not an XXX site hopefully), the type of web browser you're using (still using buggy Netscape?), etc. - you may want to redirect spinbox.net to good old 127.0.0.1 as well (which is what my HOSTS file does) to avoid being counted or probed.

    By the way, if you're wondering what kind of information can be collected on you by the mere act of browsing, go to: https://www.privacy.net/anonymizer/. What's scarier is this: do these sites try to read your cookies as well? Who knows? In the U.S., a company called DoubleClick came under fire for collecting information on web surfers and then trying to link that information to real people. Depending on how paranoid you are, there's still obvious reason for concern here.

    (Note: Apparently, you can't disable DoubleClick in the usual manner. The correct procedure is to go to https://www.doubleclick.net/optout/default.asp and follow the on-screen directions. This will place a cookie on your computer that will disable future DoubleClick tracking.)

If you wish to add to the of list of "undesirables", you can do two things: a) Try and remember any "suspect" web sites that pop on your browser status line. This is pretty hard to do though since they pop up so fast. b) Peruse the HTML source code of a site's home page. This too isn't very satisfactory since it's a hassle and also because several pages may be loaded in succession and you will only be able to look at the last one.

Sigh, I wish there was a utility which could log ALL sites/pages accessed, when you visit a site, but so far I haven't found one. I've tried some "Spyware" programs, but these tend to log only the URL that you request, not all the URLs that are associated with the site.

While the abovementioned blocking technique should work to some benefit, there are at least two instances I can think of where it's powerless to do anything:

  • First, where the ad exists BELOW the root of the server. For example, while I can redirect ads.dummy.com to 127.0.0.1, I can't redirect www.dummy.com/ads.html. The domain name must be a "dotted" name with no slashes. Granted, I could redirect www.dummy.com, but I would lose the ability to read EVERYTHING from www.dummy.com, not just their ad pages.
  • Second, where the undesirable site is referenced by an IP address rather than a domain name. The Nation's website (https://www.nationmultimedia.com) does this. Unfortunately, you can't redirect one IP address to another IP address within a HOSTS file (I think).

One final note. You can also use the above technique to prevent people from going to certain sites (same idea, right?). For example, if I were to add 127.0.0.1 www.playboy.com to my HOSTS file, no one using my computer could go there. (Of course, why would *I* want to do that?) But you get my point. Do realize though, that this is hardly a foolproof trick. A knowledgeable computer person could bypass this easily. But in certain situations, it's a cheap alternative to "baby sitter" software like NetNanny or Surfwatch.



wobble
Copyright © 2001, Thiravudh Khoman