Developing Applications with WireHose

WireHose Developers Guide

Running the import

Next, you'll add another system property to control whether or not feeds get crawled at runtime. Then it's time to test the crawler.

Add these lines to the initialize method in Application.java:

if (NSPropertyListSerialization.booleanForString(
    System.getProperty("CrawlFeeds"))) {
    Importer.crawlFeeds();
}

Add these lines to the Properties file:

# controls whether feeds are crawled at runtime
CrawlFeeds = NO

Choose Edit Active Executable 'HelloWorld' from the Project menu.
Click the + icon under Launch Arguments to add this argument, and check the Use column:
```
-CrawlFeeds YES
```

Now build and launch Hello World, and watch as the feeds are crawled. You don't need to crawl all the feeds for this example, so you can stop the application once a few have been imported. The output should look something like this:

Reading MacOSClassPath.txt ...
Launching HelloWorld.woa ...
java -XX:NewSize=2m -Xmx64m -Xms32m -DWORootDirectory="/System" -DWOLocalRootDirectory="" -DWOUserDirectory="/Users/garyt/Library/BuildProducts" -DWOEnvClassPath="" -DWOApplicationClass=Application -DWOPlatform=MacOS -Dcom.webobjects.pid=6807 -classpath WOBootstrap.jar com.webobjects._bootstrap.WOBootstrap -WODebuggingEnabled NO -NSDebugGroups "(com.wirehose.base.WHLog.DebugGroupImporting, com.wirehose.base.WHLog.DebugGroupWireHose)" -NSDebugLevel NSLog.DebugLevelCritical -CrawlFeeds YES
appRoot is /Users/garyt/Library/BuildProducts/HelloWorld.woa/Contents
Loading /Users/garyt/Library/BuildProducts/HelloWorld.woa/Contents/MacOS/MacOSClassPath.txt
Generated classpath: 
  /Users/garyt/Library/BuildProducts/HelloWorld.woa/Contents/Resources/Java/HelloWorld.jar
  /System/Library/Frameworks/JavaFoundation.framework/Resources/Java/javafoundation.jar
  /System/Library/Frameworks/JavaEOControl.framework/Resources/Java/javaeocontrol.jar
  /System/Library/Frameworks/JavaEOAccess.framework/Resources/Java/javaeoaccess.jar
  /System/Library/Frameworks/JavaWebObjects.framework/Resources/Java/javawebobjects.jar
  /System/Library/Frameworks/JavaJDBCAdaptor.framework/Resources/Java/javajdbcadaptor.jar
  /System/Library/Frameworks/JavaWOExtensions.framework/Resources/Java/JavaWOExtensions.jar
  /System/Library/Frameworks/JavaXML.framework/Resources/Java/javaxml.jar
  /Library/Frameworks/WireHoseBase.framework/Resources/Java/WireHoseBase.jar
  /Library/Frameworks/WireHoseLayoutSupport.framework/Resources/Java/WireHoseLayoutSupport.jar
  /Library/Frameworks/WireHoseWOBuilderBindings.framework/
  /Library/Frameworks/WHOpenBasePrototypes.framework/
  /Library/Frameworks/OpenBasePKPlugIn.framework/Resources/Java/OpenBasePKPlugIn.jar
  /Users/garyt/Library/Java/
  /Library/Java/
  /System/Library/Java/
  /Network/Library/Java
  /Library/WebObjects/Extensions/activation.jar
  /Library/WebObjects/Extensions/avalon-framework-4.1.2.jar
  /Library/WebObjects/Extensions/axis-ant.jar
  /Library/WebObjects/Extensions/axis.jar
  /Library/WebObjects/Extensions/commons-discovery.jar
  /Library/WebObjects/Extensions/commons-logging.jar
  /Library/WebObjects/Extensions/jaxrpc.jar
  /Library/WebObjects/Extensions/log4j-1.2.4.jar
  /Library/WebObjects/Extensions/logkit-1.0.1.jar
  /Library/WebObjects/Extensions/mail.jar
  /Library/WebObjects/Extensions/saaj.jar
  /Library/WebObjects/Extensions/wsdl4j.jar
  /Library/WebObjects/Extensions/xmlrpc-1.1.jar
  /Library/WebObjects/Extensions/
[2003-07-10 02:50:46 PDT] <main> WireHose Server 3.0 -- The WireHose frameworks are copyright 2000-2003 Gary Teter. All rights reserved. WireHose is a trademark of Gary Teter.
[2003-07-10 02:50:48 PDT] <main> Created adaptor of class WODefaultAdaptor on port 2020 and address icecube.bulldogbeach.com/192.168.0.101 with WOWorkerThread minimum of 16 and maximum of 256
[2003-07-10 02:50:49 PDT] <main> Application project found: Will locate resources in '/Users/garyt/Desktop/HelloWorld' rather than '/Users/garyt/Library/BuildProducts/HelloWorld.woa' .
[2003-07-10 02:50:55 PDT] <main> Creating LifebeatThread now with: HelloWorld 2020 icecube.bulldogbeach.com/192.168.0.101 1085 30000
[2003-07-10 02:50:55 PDT] <main> Welcome to HelloWorld, another top-quality application using the WireHose frameworks.. The WireHose frameworks are copyright 2000-2003 Gary Teter. All rights reserved. WireHose is a trademark of Gary Teter.
[2003-07-10 02:50:55 PDT] <main> The WireHose-specific defaults are:
	WHAdaptorDict = adaptorDict.plist
	WHComponentsWithContentAreStateless = YES
	WHCookieDomain = default
	WHCookiePath = /
	WHDefaultAffiliate = default
	WHDefaultLayout = Default
	WHDefaultTagEntity = WHTag
	WHDisableAutoSubEntities = YES
	WHDisableGuestPreloading = NO
	WHDisableLayoutDictionaryCaching = YES
	WHDisableSharedObjectLoading = YES
	WHHeaderDebugEnabled = NO
	WHIgnoreMissingEntities = YES
	WHLayoutDict = layoutDict.plist
	WHLookupDictionaryDebugEnabled = NO
	WHRewriteSessionCookiePath = YES
	WHSQLExceptOperator = EXCEPT
	WHSQLIntersectOperator = NO
	WHSQLTimestampFormat = default
	WHServerName = 127.0.0.1:2020
	WHServerNameHeaderKeys = ( "x-webobjects-server-name", "SERVER_NAME", "WHServerName" )
	WHStopWordsList = stopwords.txt
	WHTagCacheSize = 1024
	WHUseEntityHints = YES
	WHUserAgentHeaderKeys = ( "HTTP_USER_AGENT", "user-agent" )
	WHUserEntityName = WHUser
[2003-07-10 02:50:56 PDT] <main> WHDisableSharedObjectLoading=YES, disabled shared object loading
[2003-07-10 02:50:58 PDT] <main> EOModel 'RSSFeed' loaded... Connection dictionary replaced.
[2003-07-10 02:50:58 PDT] <main> EOModel 'RSSItem' loaded... Connection dictionary replaced.
[2003-07-10 02:50:58 PDT] <main> EOModel 'WireHoseBase' loaded... Connection dictionary replaced, URL was 'jdbc:FrontBase://localhost/wirehose/user=wirehose', is now: 'jdbc:openbase://127.0.0.1/HelloWorld'.
[2003-07-10 02:50:58 PDT] <main> EOModel 'WHOpenBasePrototypes' loaded... Didn't find WHShouldReplaceAdaptorDictionary=YES in userInfo, will not replace adaptor dictionary.
[2003-07-10 02:51:09 PDT] <main> Found 713 to crawl...
[2003-07-10 02:51:09 PDT] <main> Crawling About.com Botany: http://www.growinglifestyle.com/h117/index.rss
[2003-07-10 02:51:57 PDT] <main> Importing [RSSItem 48854d] 2003-07-10 09:51:57 Etc/GMT Concrete Countertops: Design, Form, and Finishes for the ...... 
[2003-07-10 02:52:00 PDT] <main> WireHose frameworks: Found valid license key. Unlimited transactions per minute. Non-expiring.
[2003-07-10 02:52:02 PDT] <main> Crawling About.com Home Repair: http://www.growinglifestyle.com/h108/index.rss
[2003-07-10 02:52:04 PDT] <main> Importing [RSSItem ab5e0b] 2003-07-10 09:52:04 Etc/GMT Mosquito Trap, 3/4 Acre Mosquito Catcher... 
[2003-07-10 02:52:07 PDT] <main> Crawling About.com Interactive Fiction: http://interactfiction.about.com/library/news/ifnews.rss
[Fatal Error] :34:12: Open quote is expected for attribute "NAME".
[2003-07-10 02:52:09 PDT] <main> WHImporter.fetchSnapshotsFromURL() - Error decoding root dictionary: Open quote is expected for attribute "NAME".
[2003-07-10 02:52:09 PDT] <main> Exception importing http://interactfiction.about.com/library/news/ifnews.rss - : com.webobjects.appserver.xml.WOXMLException [org.xml.sax.SAXParseException] Open quote is expected for attribute "NAME".
[2003-07-10 02:52:09 PDT] <main> Crawling About.com Interior Decorating: http://www.growinglifestyle.com/h113/index.rss
[2003-07-10 02:52:11 PDT] <main> Importing [RSSItem 24b943] 2003-07-10 09:52:11 Etc/GMT Brill Luxus 38 Reel Push Manual Mower... 
[2003-07-10 02:52:13 PDT] <main> Crawling About.com Landscaping: http://www.growinglifestyle.com/h110/index.rss
[2003-07-10 02:52:15 PDT] <main> Importing [RSSItem daa156] 2003-07-10 09:52:15 Etc/GMT Concrete Countertops: Design, Form, and Finishes for the ...... 
[2003-07-10 02:52:15 PDT] <main> [Adding tags to  1... ] 
[2003-07-10 02:52:15 PDT] <main> Crawling About.com Roses: http://www.growinglifestyle.com/h101/index.rss
[2003-07-10 02:52:16 PDT] <main> Importing [RSSItem 82fd0f] 2003-07-10 09:52:16 Etc/GMT Plants of the Metroplex... 
[2003-07-10 02:52:19 PDT] <main> Crawling Advogato: http://www.advogato.org/rss/articles.xml
[2003-07-10 02:52:20 PDT] <main> Importing [RSSItem 8dea20] 2003-07-10 09:52:20 Etc/GMT White Box Vs Black Box Voting Systems... 
[2003-07-10 02:52:20 PDT] <main> Importing [RSSItem 30b6a4] 2003-07-10 09:52:21 Etc/GMT Open Advogato?... 
[2003-07-10 02:52:22 PDT] <main> Importing [RSSItem d6ea02] 2003-07-10 09:52:22 Etc/GMT Which License for Free Documentation?... 
[2003-07-10 02:52:22 PDT] <main> Importing [RSSItem c1902d] 2003-07-10 09:52:23 Etc/GMT Forking the good fork... 
[2003-07-10 02:52:24 PDT] <main> Importing [RSSItem 7fa3f6] 2003-07-10 09:52:24 Etc/GMT Open Investment... 
[2003-07-10 02:52:24 PDT] <main> Importing [RSSItem c8092a] 2003-07-10 09:52:25 Etc/GMT CounterfeitProof... 
[2003-07-10 02:52:26 PDT] <main> Importing [RSSItem 4d75ae] 2003-07-10 09:52:26 Etc/GMT Open source software and ethics... 
[2003-07-10 02:52:26 PDT] <main> Importing [RSSItem 76358a] 2003-07-10 09:52:27 Etc/GMT UKUUG Linux 2003 conference: Early Bird registration until end June... 
[2003-07-10 02:52:28 PDT] <main> Importing [RSSItem c126b3] 2003-07-10 09:52:28 Etc/GMT Nine days before Software Patent in Europe.... 
[2003-07-10 02:52:30 PDT] <main> Importing [RSSItem a04cf8] 2003-07-10 09:52:30 Etc/GMT How should we encourage donations for software?... 
[2003-07-10 02:52:31 PDT] <main> Crawling Aerospace and Defense Industry News: http://www.moreover.com/cgi-local/page?o=rss&c=Aerospace%20and%20defense%20industry%20news
[2003-07-10 02:52:34 PDT] <main> Importing [RSSItem 4651f2] 2003-07-10 09:52:34 Etc/GMT Farm machine helps Boeing production...

Previous | Next