I've heard a number of definitions for RSS, but the one I think is standard is "Really Simple Syndication". RSS is a stand way of communicating information. It's nothing more then a standard XML document, that "RSS Readers" or "RSS Consumers" or "RSS Aggregaters" accept.
RSS for kbAlertz.com is just another way to be notified of new information on kbAlertz.com. The most common way people subscribe to kbAlertz.com is by e-mail. When we find a new article in the Microsoft Knowledge Base, we send you an e-mail. Since so many of our subscribers are technical in nature, it was a VERY common request for us to publish our notifications through an RSS feed. This is nothing more then an XML document that has the same information in it, that out email notifications do.
So for example if you wanted to get RSS notifications for ASP.NET, you would point your news reader to http://www.kbalertz.com/rss/aspnet.xml (take a look at that URL). At first glance, it looks like a screen full of text, but it's formatted in the "Standard RSS 2.0" format, so newsreader's understand the content.
The newsreader I use integrates to outlook, and is called "News Gator". So the next question is, "why do you use RSS if it comes into outlook, like the email does?". Well not all content that is available in RSS is also available by email. If fact the blogging (short for Web Logging) community is 99.99% only available by RSS.
Hopefully this clears up the question, "What is RSS?".