Log file in XML format?

| 6 Comments | 7 TrackBacks
One more logger with clunky text-based log file format and appropriate plumbing (object model, writer, parser and viewer) were written by me this week. Format was defined by customer and it was non disputable unfortunately. As for me it's just ugly. Why not use XML as log format?
Pros: trivial writing/parsing, portability, readability, simplicity. Constras: everybody seems to think it's unfeasible due to XML well-formedness and hence root element end tag problem - to append records you need to seek an appropriate place hence to parse the whole XML document. That's true for XML documents, but what about XML fragment?

XML fragment is actually external general parsed entity in XML 1.0 specification terms - it's freestanding fragment of XML, which can be incorporated into an XML document by entity reference, but it's still useful on its own - one can append elements to it (and because it's not XML document, root-level well-formedness rules don't apply to it, so one can just append elements to the end of file, without necessity to parse the whole log file) and then read such log file by XmlTextReader, which [thankfully to .NET developers] supports XML fragments (see "Reading XML Fragments with the XmlTextReader").

So here is small proof-of-concept example:

Writing to log:

class Test {
  static void Main(string[] args) {
    using (FileStream fs = File.Open("log.xml", 
        FileMode.Append, FileAccess.Write, FileShare.Read)) {
        XmlTextWriter writer = new XmlTextWriter(fs, Encoding.ASCII);
        for (int i=0; i<3; i++) {
          writer.WriteElementString("item", "", 
            DateTime.Now.ToString());
          writer.WriteWhitespace("\n");
        }
        writer.Close();
    }
  }
}
First run creates log.xml:
<item>7/22/2003 11:15:42 AM</item>
<item>7/22/2003 11:15:42 AM</item>
<item>7/22/2003 11:15:42 AM</item>
Second run appends three more items:
<item>7/22/2003 11:15:42 AM</item>
<item>7/22/2003 11:15:42 AM</item>
<item>7/22/2003 11:15:42 AM</item>
<item>7/22/2003 11:16:12 AM</item>
<item>7/22/2003 11:16:12 AM</item>
<item>7/22/2003 11:16:12 AM</item>
Reading log:
class Test {
  static void Main(string[] args) {
    using (FileStream fs = File.OpenRead("log.xml")) {
      XmlParserContext context = new XmlParserContext(
        new NameTable(), null, null, XmlSpace.Default);
      XmlTextReader reader = new XmlTextReader(fs, 
        XmlNodeType.Element, context);
      while (reader.Read()) {
        if (reader.NodeType == XmlNodeType.Element) {
          Console.WriteLine("Element: {0}, Value: {1}", 
            reader.Name, reader.ReadElementString());
        }
      }
    }
  }
}
And result is:
D:\projects\Test2\bin\Debug>Test2.exe
Element: item, Value: 7/22/2003 11:15:42 AM
Element: item, Value: 7/22/2003 11:15:42 AM
Element: item, Value: 7/22/2003 11:15:42 AM
Element: item, Value: 7/22/2003 11:16:12 AM
Element: item, Value: 7/22/2003 11:16:12 AM
Element: item, Value: 7/22/2003 11:16:12 AM 
I like it. Comments?
Doing web page design on your own can be as successful as your web site design coming from a professional web designer but you'll want to make sure that your web site design is user-friendly and clean before worrying about bells and whistles.

Related Blog Posts

7 TrackBacks

TrackBack URL: http://www.tkachenko.com/cgi-bin/mt-tb.cgi/55

TITLE: Reading XML fragments with XmlTextReader URL: http://weblogs.asp.net/cazzu/archive/0001/01/01/119263.aspx IP: 66.129.67.202 BLOG NAME: kzu.net DATE: 04/24/2004 07:22:56 AM Read More

TITLE: Reading XML fragments with XmlTextReader URL: http://weblogs.asp.net/cazzu/archive/0001/01/01/119263.aspx IP: 66.129.67.202 BLOG NAME: kzu.net DATE: 04/25/2004 02:24:11 AM Read More

TITLE: Mvp.Xml project: Reading XML fragments with XmlTextReader - XmlFragmentStream URL: http://weblogs.asp.net/cazzu/archive/0001/01/01/119263.aspx IP: 66.129.67.203 BLOG NAME: DATE: 04/28/2004 02:31:45 AM Read More

TITLE: Reading XML fragments with XmlTextReader - XmlFragmentStream URL: http://weblogs.asp.net/cazzu/archive/0001/01/01/119263.aspx IP: 66.129.67.202 BLOG NAME: DATE: 06/22/2004 07:34:57 PM Read More

TITLE: RE: Appending XML files and confusing disposables URL: http://weblogs.asp.net/jvdbos/archive/2004/06/28/167672.aspx IP: 66.129.67.203 BLOG NAME: J e r o e n ' s DATE: 06/28/2004 12:56:37 PM Read More

TITLE: Reading XML fragments with XmlTextReader - XmlFragmentStream URL: http://weblogs.asp.net/cazzu/archive/0001/01/01/119263.aspx IP: 66.129.67.202 BLOG NAME: DATE: 07/09/2004 03:11:23 PM Read More

And you thought XML is done? No way. It's alive and kicking technology. And here is just one more proof: yet another new XML API from Microsoft - the XmlLite. It's a native library for building high-performance secure XML-based applications. XmlLite li... Read More

6 Comments

Sumit, first of all you are wrong when you are saying that absense of root element makes XML invalid. Not at all. XML document must have root element, but XML fragment AKA external general parsed entity - not.
Second - yes, you can always write a parser for a custom log format, but I think that's just stupid waste of time. Just log in XML and forget about writing custom parsers, get over this "anyone who doesn't write in asm is an idiot" attitude, save you time to developing something actually useful.

You've got to be kidding me? XML? For a log file?

First, XML is just overused in everything. It's a horribly over-verbose format that adds bloat to the size of any document.

Second, how would you do rolling logs? Apache has an XML log writer in log4j. It contains no root element. It just puts in chunks of XML..with no root element. So the XML is invalid. It can never be read by a standard parser that does validation. (your code also generates just such an invalid XML log) You could close off the XML when the log rolls over, but ... seriously? Why are we ever talking about this?

Logs should simply be one line, one log entry. It's very easy to parse good logs with a good scripting language such as PHP, Python, Ruby, etc. without the need of a huge clunky god awful XML parser. Your code does nothing but add a retarded "" tag around each line. What's wrong with using the newline character to tell when the end of a line is.

Anyone who uses XML to log anything is an idiot.

Sumit Khanna

Thnks

very good

Sounds interesting. Any URL?

We went a little bit further with that approach by adding ability to emit hierarchical logs (e.g. logging specific operations in scope of parent operations etc.). This enabled some interesting ways of analyzing those hierarchical logs.

Leave a comment