Dreams of a Rarebit Fiend

HotSheet has gone on a diet and is now down to just a little over 500K. That makes for a nice quick download even for dial-up users. I'm not going to claim that it's not going to gain some weight back soon but I've been cleaning up the code with an eye to a lot of new development and also trying to keep its size in line with the amount of functionality the end user is getting.

The best part is that I'm approaching the end of my cleanup process and I'll be starting on adding new features and making the whole program easier to use next week.

Posted by John Munsch at 05:31 PM | Permalink | Comments (0)

I just looked at CVS for HotSheet and I haven't checked anything in since July of last year. Which sounds about right because I made some mistakes in writing it initially and then I got it to a state where it kind of limped along well enough for me to use it every day so there wasn't a huge drive to do all the improvements it needs.

But I'm finding myself annoyed by the problems it has and I'm wanting it to start realizing more of its potential so you can expect a new version soon. It will have at least the following changes:

• The item history is currently staying around even if you delete the item history file. This odd quirk is a side effect of my using serialization to save off my internal structures and a minor bug in my code playing off of each other. [100% done]
• The default browser that launches for Unix/Linux users is now Mozilla. Since most users run the software through Java Web Start, this won't change things an iota for you. Only if you were doing development with the code and running it from the command line or from the Ant buildfile. [100% done]
• Many channels were including HTML tags right in their RSS entries. This made the results almost impossible to read when you saw them rendered in HotSheet. New code in the RSS library allows for getting either the raw descriptions or the description with all tags removed and HotSheet now uses the latter. [100% done]
• Prevayler will be used for a new version of the item store, channel store, and item history. This will make a huge difference in HotSheet's ability to deal with crashes or other problems. Fortunately, due to how the first version of the RSS library was written, this isn't going to be too big a deal. [0% done]
• I went on a mission to cut as much as possible from the download so it would be a quicker download and I wouldn't expend as much bandwidth for new people trying it out. I dropped quite a bit by removing all the XML libraries that are now part of the Java Runtime and even more by dropping the scripting support that isn't even properly incorporated anyway. As a result it should be a lot trimmer and faster to install than before. If I was willing to cut my usage of Log4J and instead substitute the J2SE 1.4 logging I would have cut it from 4.2MB to .2MB. But I'm not willing to do that so you'll have to settle for a .4MB download instead. [100% done]

Posted by John Munsch at 02:37 PM | Permalink | Comments (0)

I've had the "pleasure" of moving from Windows to Linux at work for the first time ever this week and while it is helping me remember how to do stuff that I last did in school 15 years ago, it is also teaching me much about the current state of Linux for end users.

Anyway, one of those lessons had to do with sound on Linux. Linux has several schemes for sharing the audio hardware and unfortunately Sun chose to not support any of them, though they say they might for the 1.4.2 JVM. A Java program on Linux simply cannot play sound if anything else is using the sound hardware. As a result there is a new release of HotSheet available that does not play a sound when it finds new items during a refresh. If you add "sound.newItems=true" to your HotSheet.properties file then it will play sounds again (for Windows and Mac OS X users).

Posted by John Munsch at 08:29 AM | Permalink | Comments (2)

I can't speak for Solaris or XP but the bug where HotSheet wouldn't come up when run with JDK 1.4.0 has been fixed as far as Linux is concerned. Run it again and you'll get an updated version which should run without incident.

Posted by John Munsch at 05:54 PM | Permalink | Comments (1)

Here is the email I posted to the HotSheet mailing list the other day. I'm pretty sure the readership of the list is pretty darn low but surprisingly enough, this site gets way more traffic than you would expect so I'm repeating it here.

I'm working on HotSheet again. I've done a few small updates to the source over the last month but no big changes that warranted a release. Now I'm trying to get the dialog done that will allow adjusting the various properties that could previously only be set through the properties file. I did a lot of work last night on improvements to the RSS library and I'm hoping to bang on the program for at least an hour a day every day until I see some big improvements.

So now that I use HotSheet on a daily basis and I'm no longer sick of looking at it I've started to ask myself what the future holds for it. The conclusion that I've come to is that RSS is not enough. News headlines are nice but wouldn't it be better if there were a set of data classes that handle RSS (obviously), but also weather, daily comics, sports scores, other forms of news, email notifications, calendar event notifications, stock info, tv shows, and tracking of items that change status like auctions or shipped packages. In other words, all the things people go to portals for all the time. People could build anything from a portal to an information appliance to a web service. By having one library there could be more consistency in interface, documentation, testing, etc. and the results would probably gain a lot more attention than a scattered set of separate libraries to do the same things.

I've got ideas where to graft on existing libraries to get some of this data:

Weather
http://www.amug.org/~glguerin/sw/#weather
http://javaweather.sourceforge.net/
Sports scores
http://sportwire.sourceforge.net/
Stocks
http://www.neuro-tech.net/app.xml?name=jfl
Email notifications
http://java.sun.com/products/javamail/index.html

but for others there isn't an existing Java solution. We might port an existing application in another language or wrap the data that it pulls. Some likely candidates for that treatment include:

Comics
http://sourceforge.net/projects/pycomic/
TV
http://noomore.org/GuideParser/
http://freeguide-tv.sourceforge.net/ (this is an existing java project that uses data pulled via Perl)
http://membled.com/work/apps/xmltv/

Still others, like auction and package tracking... I've got no knowledge of existing libraries to use, and for some of the existing ones mentioned above there may be license issues to be considered. It's very important to me that any library included be either BSD, LGPL, or some equivalent that won't impose unpleasant restrictions on any application in which it is used. Especially since it may represent a very small part of functionality in the application.

Clearly this kind of expansion will require me to separate HotSheet into two projects rather than just one. There will be a data libraries project and a separate project or projects for interfaces to that data.

In the meantime, expect a new release of HotSheet within the next couple of weeks.

Posted by John Munsch at 03:59 PM | Permalink | Comments (1)

Hey! Sun mentioned HotSheet on their website. They have a kind of newsletter like posting periodically to show off some of the neater applications written with Swing, their cross-platform user interface framework. I used it when I wrote HotSheet and they included it in their latest roundup.

Posted by John Munsch at 11:07 AM | Permalink

No program is ever really finished. You work on it and you work on it until you get tired of it and abandon it or maybe you never get tired of it and you never stop working on it. I suppose some really trivial programs get finished. Ones that just read a file and spit out another version of that file or something. But I've written few that fall into that category, most of my programs are never finished.

HotSheet falls into that category. I'm looking at it and all the work that still remains to be done on it and I just can't believe it. I am going to do another release of the source code soon but it will probably be the first one I've done where I knew there were bugs in it and I needed to just make it available because it was a big improvement over what is already out there, bugs or not. I'm tired of working on it, but if I stop there is no guarantee that anyone else will ever take it over and finish it, and it needs to be finished to really be useful to people. Sigh...

Posted by John Munsch at 02:04 PM | Permalink

The source code for HotSheet has been downloaded exactly 300 times. Some of those were doubtless the same people downloading different revisions but nevertheless we can safely say that 200+ people have probably wanted the source at one time or another.

Is anybody doing anything with it? I have to wonder because nobody ever contacts me about it. I know that technically they don't have to, they could go ahead and incorporate it into a commercial program and not give me the time of day. But I'd love to know if somebody, anywhere found some use for the RSS library portion of it or is trying to rip some other part of it out or learn something from it.

Posted by John Munsch at 01:31 PM | Permalink

There's a new release of HotSheet out. It should auto-download and install whenever you next run it but I did encounter an error on one machine where I had to try running it a couple of times before it downloaded. I have no idea why.

The new release displays its progress on the status bar at the bottom as it pulls news channels. Unless people come up with bugs, this will be considered the next official version and a source release will be done soon.

Posted by John Munsch at 09:37 AM | Permalink

There is a new version of HotSheet available via the automatic update mechanism. It's not really the next version yet because the UI updates aren't finished. But it does have multi-threaded retrieval of channels using a thread pool built in. In my tests so far this has greatly sped up how long it takes to perform a "Refresh All Channels" command and it should scale even up to a program which retrieves thousands of channels.

So, what has to happen now before a new release? I've got to fix the UI so it shows the progress of channel retrievals and that is going to require careful updates to the RSS library. I've done some already but I want to review what I did to make sure it seems consistent with the rest of the library. After that I need to see if I can use Harshad Oak's code to give us a broader set of channels to pick from. Then I'll do a release before I reopen the patient again to add scripting and fix up the messy item history system.

By combining scripting and some new methods for marking news items or separating them from the rest of the items users will have the ability to start tailoring HotSheet to their own particular needs. I'm toying with a few different ideas but one that is sure to be implemented is the idea that each item will have its own set of properties. That way I can tag items as "read" and they will stay tagged. To the library the properties will just be strings and numbers associated with the items, it won't have any idea how they are used. It will be up to the program that uses the library how it interprets different properties.

Given that capability it will be easy to add a "score" to each news item and make HotSheet aware of scores on news items. Scripts could review news items and create a score based on keywords they see or other characteristics that will be attached to the item. HotSheet will draw items that have a score property differently than ones without, draw them differently based on how high the score is, or perhaps place them on a different tab. I've been thinking that it would be neat to have separate tabs in the UI with all the news items on one tab and the "hot" news items on another. Anyway, suggestions are welcome because I'm just trying to think what kind of abilities I'm going to want to make available to the scripting languages that can be connected to HotSheet.

Posted by John Munsch at 01:49 PM | Permalink

I've been working on a new release of HotSheet the last few days. I had originally planned to have it ready by Friday but now it's looking like it won't even be done until Friday so a few days of testing will push it into next week. On the plus side though, one of the developers has already done some work that will help in two areas:

1. It will give us a big list of channels to choose from. This is really important for the popularity of the project because it will mean that the average person can use it simply by selecting some channels that interest him or her.
2. I can use it to test the compatability of the RSS parsers with the channels that are out there. After I get the new code I'm going to use it to try and parse every single channel in our new list (which will include hundreds of channels). The results of that test will tell me where I can look to find bugs in the parsers and also, which subset of channels we can include in our initial list.

Speaking of Virtual Cardboard, doesn't that name just suck? I've been trying to come up with something better but so far I haven't gotten any inspiration.

Posted by John Munsch at 01:19 PM | Permalink

"Where's the source John?"

That's a fair question considering that I uploaded a new version of HotSheet Thursday but haven't released the source for it yet. I delayed thinking I could get something in the build file that would automatically increment a build number every time there was a successful build. Unfortunately, that looks to be a little harder than I thought. Then I vegged over the weekend and went to the Dave Matthews concert last night. So I accomplished absolutely zero this weekend.

I'll just zip everything up and put it out there tonight. I promise. It won't have auto-incrementing build numbers but that's not exactly critical. More important will be utilizing the new events/listeners to change the way channel refreshes are done. Currently everything is very sequential and linear: retrieve a channel, parse the channel, update the model, go to next channel, repeat.

That's great when everything is working, but you have problems like the one today where BetaNews seems to be down. They aren't serving up the page that HotSheet expects and since I currently rely on the JAXP code to do both the retrieval and parsing I don't have any way to tell it to just timeout if the URL can't be retrieved in a timely manner. As a result HotSheet effectively hangs waiting for something to happen on that channel and never gets to other channels that it could actually retrieve.

Posted by John Munsch at 12:50 PM | Permalink

Next up will be to make use of the newly added events and listeners code to: play a sound when new items are added, hook up a timer to refresh channels automatically and make the interface update whenever new items are added.

Then I'll tackle using HypersonicSQL to provide a database to hold all your items, channels and history rather than the serialized linked lists and hash maps that are being used now. It's not as simple and convenient but it will be much much faster, especially when you have lots of items and channels. With HypersonicSQL I'll be able to bundle a JDBC database with the program to provide built in capabilities but for users who want to use the program and have it automatically populate another database with data (for example to populate a database that a website then queries to display news items) you'll be able to use it with other JDBC drivers against databases like mySQL or Oracle.

Posted by John Munsch at 11:14 PM | Permalink

There's a new release of HotSheet on the server (upgrade is automatic next time you run it). It has several big changes:

• News items are automatically filtered out and not added to the list if they've been added before. You can now refresh without clearing the list and not get duplicates!
• The released version will show both the current version number and the build number in the about box.
• (For Developers) There is a standard Java events and listeners system in the RSS library now. You can listen to events on the ItemStore and ChannelStore and just react to them if that is a more natural model for your program. There is also a new demonstration program that shows events and listeners and demontrates the basics of using the item and channel stores.
• (For Developers) The demonstration programs have been broken out into their own jar to make HotSheet a smaller deployment.

Posted by John Munsch at 11:07 PM | Permalink

I've updated the project page for HotSheet. It now shows screenshots of HotSheet running on Windows and Mac OS X (thanks Morbus!) as well as updates to some of the text on the page and a link to OpenJNLP for Mac users.

Finally bought a digital camera and it's not one of the fancy ones I had been planning to buy. Originally I had planned on buying either the Nikon Coolpix 990 or the Olympus 3030. Both are 3.3 megapixel cameras with loads of features and very high image quality. Unfortunately both are also more than $600 and Rockelle and I are finding ourselves frustrated by not having a real digital camera (my son's JamCam toy not counting). So instead the initial winner is the Kodak DC 3200 1 megapixel camera which sold for $129 from OfficeDepot. After having used it the only think I can complain about is the lack of a USB connection to the camera (which oddly enough, the JamCam has). Downloading images over serial is really pretty dark ages but it won't be an impediment for long because it takes CompactFlash cards and we intend to get one and a USB reader (probably from Lexar) to expand the number of images we can take and speed their download significantly.

Posted by John Munsch at 11:33 PM | Permalink

Long time, no web log. I'm not dead, I've just been busy reading, working, programming, etc.

HotSheet has had events and listeners added to it and I'm in the process of building a demo application which will demonstrate the addition and test it. This addition will make it a lot easier to build applications which react to new items coming in rather than having to drive the whole process. That's a much more natural model for when HotSheet or any other application which uses the RSS library is timer driven.

Posted by John Munsch at 06:19 PM | Permalink

HotSheet has been updated to mark items you have already viewed (they now appear ghosted) and menu options have been added to let you delete all viewed items or all selected items. That closes a couple of the wish list features for HotSheet, only 17 or so to go :)

Posted by John Munsch at 12:15 AM | Permalink

Fixed a bug in HotSheet that was causing it to flip out on one of the default channels. Since I had the code open already while I was trying to add another feature, I had to hurry up and finish it so I could upload the fix. As a result, Windows users should now have mousewheel support. If the software suddenly stops working for Linux, Solaris, etc. users please email me. I can't test platforms other than Windows so I may not know that I broke everybody's software.

As always, the upgrade is an automatic one the next time you run HotSheet. Source code changes will be uploaded shortly to SourceForge for general availability.

Posted by John Munsch at 09:28 PM | Permalink

Well, apparently there are some people interested in HotSheet. It managed to hit the 90% activity level today on SourceForge which makes it the 169th most active project (as measured by downloads, clicks, etc.). That's gratifying. Now as a friend of mine observed when I explained the project to him today, the software "isn't exactly rocket science." No, it's not rocket science :), but it is a good start toward something that will gather news and make it available in helpful ways.

SourceForge responded to my bug report about problems with posting a new file by posting it for me, but my bug report stands. I hope that means that they are still intending to address the underlying problems I had with posting the file in the first place and they aren't just doing a band-aid fix to alleviate one particular instance of a wider problem in the file release system. Note: This may sound like looking a gift horse in the mouth. It is not. SourceForge is neat, it's free, and it's very very helpful. I just look forward to the day when they've squashed a few more of their bugs and they aren't modifying the software quite as much on an active system. It is easier to develop software that isn't always perfectly stable if the platform you are doing development on is stable.

Posted by John Munsch at 10:51 PM | Permalink

Various bug fixes and corrections to HotSheet based on user feedback:

1. The instructions for running HotSheet from the command line (mainly intended for developers working on the code) have quotes around the classpaths. This works better on Mac OS X and retains compatability with Windows. (Thanks to: Aaron Swartz)
2. A confusing error message that was issued whenever the channel list isn't found (like the first time you run) has been replaced with something that is actually informative. (Thanks to: Aaron Swartz)
3. Double clicking on a news item wasn't working everywhere. I changed the program to use the Java Web Start function to load a browser and display a page when the program is run under Web Start. It will still fall back to the old system when run from the command line though. One small downside is that the Web Start function always opens a new browser to display a given URL, it would be great if they had a flag that let you choose which you wanted but Sun doesn't provide it. (Thanks to: Richard Katz)

As always, anyone who installed the program via Java Web Start will be automatically upgraded to the new version. Source will be available momentarily on SourceForge as HotSheet 0.51 Alpha.

Posted by John Munsch at 01:58 PM | Permalink

Friday was very busy. I am pleased to announce that the source code to HotSheet 0.5 Alpha is available on SourceForge! Last time I checked three people had downloaded it already. Maybe nobody will do anything with it but I can always hope. For my part I'm going to keep adding to it periodically to improve what it can do until it reaches the level of commercial programs like Novobot and Headline Viewer.

Posted by John Munsch at 01:04 PM | Permalink

I got approval over the weekend for a HotSheet project on SourceForge. I've started the moving in process but there's still quite a bit to be done.

In the future, all development on the project will be done through SourceForge but a stable version of the program will always be available from the project page on this site.

Posted by John Munsch at 12:47 PM | Permalink

I've done work in two areas toward release of the source for HotSheet:

1. Started writing the developer documents.


2. Finished a second demonstration application that uses the RSS libraries in HotSheet. The demonstration applications plus the developer docs should be enough for people to get started building new applications using the source.

Last night however I spent a couple of hours watching and fast-forwarding through about six hours of TechTV's coverage of the Electronic Entertainment Expo (E3). E3 is where lots of game stuff happens every year and this year is a bumper crop of news about what is happening in the console arena.

I watched about a half hour of the Microsoft press conference to unveil the Xbox. It has truly gorgeous graphics and it makes me salivate at the prospect of someday having something like a GeForce 3 (an nVidia graphics chip used in the Xbox) for my PC someday. You can buy one currently but all the boards based on the chipset are in the $500(US) range and that's way too high for me. It's just a matter of time before they come down to a level where I can afford one though.

But graphics alone do not make a console successful and the games that I saw demonstrated for the Xbox left me cold. There was another game in the Oddworld series and although it was pretty and smooth, it didn't thrill me. The first person shooter Halo had numerous framerate stutters during their demonstration. Maybe all that will be ironed out by the release but they are under tremendous time pressure to make sure that title is ready for the Xbox launch date on November 8th.

Aside from announcing the launch date and showing off various titles that will be available on the shipdate Microsoft was also announcing their price point of $299(US). That's not a big surprise as that's exactly what a Playstation 2 costs with similar features. Nintendo's GameCube will probably be under $200(US) and they will ship November 5th but because of its hardware will not be able to play DVDs as both the Xbox and Playstation 2 can.

For me the big surprise of the day was watching the Nintendo press conference and expecting very little going in but coming away impressed with the games I saw on the platform. Luigi's Mansion actually looks like a lot of fun. Super Smash Brothers was fun on the Nintendo 64 and an updated version of it will likely be fun too. Perhaps I would have been equally impressed by what I saw from Microsoft if I had had the low expectations I had when I started watching Nintendo's.

Last entry of the day: Adobe Acrobat for the Palm OS (beta)

Posted by John Munsch at 01:05 PM | Permalink

I've spent a little time working on example programs that use the RSS reading/parsing portions of HotSheet. HotSheet is designed using Model-View-Controller (MVC) and the model portion of it is the portion that deals with RSS channels and the items within them.

I want to get some simple example programs and documentation ready so other developers could utilize parts of HotSheet to build other programs. For example, Don wants to use the core to build something that would generate a portal page that has all his favorite RSS channels on it (basically the purpose that my.netscape.com used to serve).

As soon as I get the developer docs done I plan to post everything to Sourceforge so others can use this stuff if they are interested in doing so.

Posted by John Munsch at 11:44 AM | Permalink

Added support for RSS 1.0 format to HotSheet. Now you can get news from Slashdot and other sites that use 1.0 rather than the older but more prevalent 0.90 and 0.91 RSS standards (e.g. GameDev.net still uses 0.91). As always, the upgrade will happen transparently through Java Web Start.

Posted by John Munsch at 11:27 AM | Permalink

Added the fix for the missing Netscape DTD file. You can once again run HotSheet and it will retrieve RSS files from around the web. Yay!

Posted by John Munsch at 10:29 PM | Permalink

Eric Raymond has a new editorial titled, "Why Python?" I have been interested in Python for a while now because so many programs need a scripting language built in to give them that next level of functionality. Python seems popular, easier to learn and use than Perl (which I've always thought was just one step above APL in terms of being cryptic), and it looks to be easy to interface to Java including one implementation of Python that is written completely in Java. Thus it seems like a tool I need to add to my chest.

The other thing to write about today is that I've got a pretty good idea how to fix my HotSheet code but now I have to actually do it. As best I can tell the answer is to replace the entity resolver that the JAXP parser uses. When it requests the traditional Netscape URL for the DTD, I will instead create a new input source using a copy of the DTD I'll include with every copy of HotSheet. That input source will then be supplied. If it works then I would be able to immediately begin parsing all the RSS files again whether they've updated to point to a new DTD or not.

Posted by John Munsch at 03:28 PM | Permalink

Greatly improved the rendering of items in HotSheet. Long descriptions are now wrapped and displayed on multiple lines and titles appear in bold so they stand out from the descriptions. If I get RSS 1.0 support in there (already started) and a filtering system so that refreshing the channels you have in your list doesn't make new items if there are already items that are identical still in the list then I'm going to call it functional for a little while and work on another project before coming back to it. I'm thinking about making all the source available through Sourceforge along with information about the improvements I have in mind. Perhaps someone else will want to do some of them or will have some other cool stuff in mind and enough time to improve it.

Posted by John Munsch at 02:42 PM | Permalink

Wow. Busy weekend, busy Monday.

I had family over for Easter weekend but I still managed to get HotSheet to a somewhat crude alpha state. If you look up at the top of the page you'll see that there is now a projects page listed. If you go there you will see a link to the HotSheet project page. I'd love to get feedback from anyone who tries to install and use HotSheet. If everything goes the way it is supposed to then Java WebStart will make it as simple as clicking on a link for you to be able to use HS.

If you are sharp-eyed you might also notice that most all of the formerly static HTML pages have gone to JSP. I intended to do that anyway for many of the pages eventually but my hand was forced by how difficult it is to change things within Blogger. If I just wanted to add a new menu item to the top of all the pages I had to change two different templates within Blogger and then tell it to republish my main page and all of the archive pages. The other day when I tried to do that, it just timed out again and again without ever publishing anything. So now all of the JSP replacement pages import the page header so I can change it easily.

Posted by John Munsch at 11:39 PM | Permalink

Again, quite a few days without an update here. But, that doesn't mean I've been idle. I almost have a functional version of HotSheet ready. It can parse and display the contents of RSS 0.90 and 0.91 channels already. There's still lots and lots of work to go but I think by Monday I'll have some code I can give to Don to start playing with. By late next week I think I'll have an application that you can download from this site (single click Java WebStart install of course :) that has most of the functionality of Headline Viewer or Novobot. Those are the only two end user tools that I know of that deal with retrieving RSS syndication files and displaying the news items found.

Posted by John Munsch at 02:12 PM | Permalink

Not a lot of updates this week doesn't mean that I've been totally idle. I've managed to do at least some work every day on HotSheet. HotSheet is a program to retrieve syndicated news from websites in RSS format, parse it to extract the individual items, and put all the new items into a store. That store can then be used by other parts of the program to email new items to you or display them on a website or pretty much anything your heart desires.

Posted by John Munsch at 10:45 AM | Permalink