Uniquing items

Next you'll tell WireHose which attributes should be used to distinguish one item from another during importing. This is important since new items will typically be added to the top of the feed, pushing older items off the bottom. If Hello World's crawler didn't have a method for determining if it had already seen an item, the database would be cluttered with duplicates.

In this case, duplicate items have either the same name or link.

1. Select the RSSItem entity.
2. Choose Inspector... from the Tools menu.
3. Select the EOEntity UserInfo pane.
4. Click Add.
5. Set the key name to WHKeysForUniquing, and its value to (link, name).

   

Note: The RSS 2.0 specification allows feed providers to provide an optional attribute for each item, called "guid". This globally unique identifier is specifically intended help aggregators determine if an item has been seen previously, but not all feeds use it. For simplicity, we are ignoring this attribute in this tutorial.

Copyright ©2000-2003 Gary Teter. All rights reserved. WireHose is a trademark of Gary Teter.