Jump to content


T7: Removing Duplicates From Your Proxy List


  • You cannot reply to this topic
No replies to this topic

#1 T Rex

    T Rex

  • PN Admin
  • PipPipPipPipPip
  • 1,861 posts

Posted 17 August 2004 - 12:29 PM

Neither AccessDiver or GeoWhere can eliminate all duplicates from your proxy lists, if they, your lists, contain both IP Addresses and Host Names.

Reason:

The Proxy Analyzer modules in these programs only compare the infomation in the "Proxy Server" column (AD) and "Proxies" column (GW) when attempting to find duplicates. As a result, if your lists contain both the "host name" as well as its "IP address", then you have a duplicate. See example as follows:

sdsl-64-32-210-142.dsl.chi.megapath.net:80 (this is a proxy host name)
64.32.210.142:80 (this is its resolved IP address)

To eliminate one of them using AD or GW you will first need to "resolve" your entire list by running at least a speed/ accuracy test and then save your list in "IP Address" format only. Then (of course) remove your loaded list, load your newly saved "IP Address" only list, and then ask it to remove the duplicates.

(Note: there surely are other ways to handle this, but I believe this one should be fairly easy to understand).

You could, of course, also use other programs to eliminate duplicates, but most would still require the entire list either be in host name or IP address format only.

AATools, for example, is one exception as it automatically resolves host name proxies and then compares them (the resolved IP Addresses) against the entire list. Should it find a duplicate, it then labels it as such.