Get Control Of Your Enterprise Library Logging Application Block Logs

Enterprise Library’s Logger is powerful and convenient.  It’s default TextFormatter provides a suitable human readable format, and it also allows you to implement custom formatters to suite your needs.  I think that being able to work with log data without having to manually parse strings, can really simplify the development of tools for mining logs.   To this end, I whipped up a small WinForms app that demonstrates how to take log entries originating from a text spool, convert them into objects, and query them with Linq.  The primary goal was to provide simple UI that enables log search, filter, and find, with sorting, grouping, etc.

First off, the heavy lifting in this sample is provided by the EntLib Contrib LogParser.  Their FilteredLogReader class drives a tokenizer, which creates and populates LogEntry objects out of log text.   The tokenizer will use any formatting template, in our case, I’ve used the built in “Text Formatter”.   When it executes, it generates a RegularExpression capable of matching and trapping the values involved in populating our collection of LogEntry objects.  Ultimately, we are provided with a simple API for driving all of this.  Here’s a brief sample:

string configFile = "abc.config";

            string traceLog = "trace.log";

 

            FilteredLogFileReader reader =

new FilteredLogFileReader(traceLog, configFile,

                        new TimeStampParser(CultureInfo.CurrentCulture));

 

            Filter filter = new Filter

            {

                StartTime = DateTime.Now.AddHours(-1),

                StopTime = DateTime.Now

            };

 

            IEnumerable<LogEntry> entries = reader.ParseLogFile(filter);

The basic idiom here is:

1.       Get a FilteredLogFileReader

2.       Setup a Filter

3.       Feed the filter to the reader’s ParseLogFile

To facilitate a flexible set of filtering concepts at the UI, I decided to use a simple FilterExpression class to represent a set of keys, values, and match operators.  Here’s a quick way to get the intersection result set from applying a set of filters with a single pass through the logs.

       

        private IList<LogEntry> FilterLogEntries(List<LogEntry> lst,

                                                 List<FilterExpression> exprs)

        {

            var matchAllExprs = new List<LogEntry>();

                       List<PropertyInfo> props = typeof(LogEntry).GetProperties().ToList();

            lst.ForEach(entry =>

                {

                    int matchCount = 0;

                    exprs.ForEach(expr =>

                    {

                        string value = props.Find(prop => prop.Name.Equals(expr.Key))

                                                      .GetValue(entry, null).ToString();

                        switch (expr.Operator.ToLower())

                        {

                            case "contains":

                                if (value.Contains(expr.Value))

                                    matchCount++;

                                break;

                            case "!contains":

                                if (!value.Contains(expr.Value))

                                    matchCount++;

                                break;

                            case "regex match":

                                if (new Regex(expr.Value).IsMatch(value))

                                    matchCount++;

                                break;

                            case "==":

                                if (expr.Value.Equals(value))

                                    matchCount++;

                                break;

                            case "!=":

                                if (!expr.Value.Equals(value))

                                    matchCount++;

                                break;

                            default:

                                throw new Exception("operator not found");

                        }

                    });

                    if (exprs.Count == matchCount)

                        matchAllExprs.Add(entry);

                });

            return matchAllExprs;

        }

 

If any of the LogEntries match all of the expressions, then those are returned to the caller.  Now we can place the log data onto our screen for users.

this.dataGridView1.DataSource = 

    new BindingList<LogEntry>(FilterLogEntries(lst,

        lbxFilterExpressions.Items.Cast<FilterExpression>().ToList()));

The sample app that I’ve included, gives users the ability to create, configure, and apply FilterExpressions on the fly.  Here’s a screenshot.

You can get the source for this sample here.  Enjoy..   

Object Bakery – Fun with .NET Serialization and Crypto: Part 2

In my last article, I demonstrated how to use .NET serialization to dehydrate and rehydrate objects with helpers that drive XmlSerializer, DataContractSerializer, SoapFormatter, and BinaryFormatter.  This is part 2, where I expand upon these concepts demonstrating how to use the DESCryptoServiceProvider to securely save off your objects for later consumption.   For this, I created a few helpers that encrypt an object stream and save it to disk, as well as the reverse.  Only instead of using MemoryStream as we did in the last article, we use the CryptoStream and a shared key.   Here is a test that shows a secure round trip that saves an encrypted Product instance to disk and reads it back out.

        [TestMethod]

        public void CanEncyptToDecryptFromDisk()

        {

            string strKey = ObjectHelper.GenerateKey();

            ObjectHelper objectHelper = new ObjectHelper();

            FileInfo file = new FileHelper().GetTempFile();

 

            List<Asset> assets = new TestAssetFactory().Build();

 

            objectHelper.EncryptObjectToDisk(file.FullName, assets, strKey);

 

            List<Asset> assets2 =

               (List<Asset>)objectHelper.DecryptObjectFromDisk(file.FullName, strKey);

 

            Assert.IsNotNull(assets2);

 

            file.Delete();

        }

A notable difference from the samples in part 1 is that we generate a key first and then use it to symmetrically encrypt and decrypt our persisted object.   The single assertion here is not particularly impressive, so here is a test that runs the same idiom with a DataSet and then compares the values of each field in each row for match fidelity.

        [TestMethod]

        public void CanDehydrateRehydrateDataSetToFile()

        {

            string dataKey = ObjectHelper.GenerateKey();

 

 

            FileInfo file = new FileHelper().GetTempFile();

            DataTable productsDataTable = new TestTableFactory().Build();

            DataSet productsDataSet = new DataSet("Products");

            productsDataSet.Tables.Add(productsDataTable);

 

            ObjectHelper objectHelper = new ObjectHelper();

            objectHelper.EncryptObjectToDisk(file.FullName, productsDataSet, dataKey);

            DataSet productsDataSet2 =

                (DataSet)objectHelper.DecryptObjectFromDisk(file.FullName, dataKey);

 

            Assert.IsTrue(productsDataSet2 != null);

 

            for (int j = 0; j < productsDataSet.Tables["Products"].Rows.Count; j++)

            {

                DataRow pRow = productsDataSet.Tables["Products"].Rows[j];

                DataRow pRow2 = productsDataSet2.Tables["Products"].Rows[j];

 

                for (int i = 0; i < pRow.ItemArray.Length; i++)

                {

                    Assert.IsTrue(pRow.ItemArray[i].ToString() ==

                       pRow2.ItemArray[i].ToString());

                }

            }

 

            file.Delete();

        }

 

Finally, here is the ObjectHelper class demonstrated above..

    public class ObjectHelper

    {

        public static string GenerateKey()

        {

            // Create an instance of Symetric Algorithm. Key and IV is generated automatically.

            DESCryptoServiceProvider desCrypto =

                 (DESCryptoServiceProvider)DESCryptoServiceProvider.Create();

 

            // Use the Automatically generated key for Encryption.

            return ASCIIEncoding.ASCII.GetString(desCrypto.Key);

        }

 

        public bool EncryptObjectToDisk(string fileFullPath, object data, string eKey)

        {

            FileStream fsEncrypted =

                 new FileStream(fileFullPath, FileMode.Create, FileAccess.Write);

            bool saveSuccess = false;

            try

            {

                DESCryptoServiceProvider des = new DESCryptoServiceProvider();

                des.Key = ASCIIEncoding.ASCII.GetBytes(eKey);

                des.IV = ASCIIEncoding.ASCII.GetBytes(eKey);

                CryptoStream cryptostream =

                     new CryptoStream(fsEncrypted, des.CreateEncryptor(), CryptoStreamMode.Write);

                BinaryFormatter bin =

                     new BinaryFormatter(null, new StreamingContext(StreamingContextStates.File));

                using (StreamWriter streamWriter = new StreamWriter(cryptostream))

                {

                    bin.Serialize(streamWriter.BaseStream, data);

                    streamWriter.Flush();

                    streamWriter.Close();

                    saveSuccess = true;

                }

            }

            finally

            {

                if (fsEncrypted != null)

                    fsEncrypted.Close();

            }

            return saveSuccess;

        }

 

        public object DecryptObjectFromDisk(string fileFullPath, string eKey)

        {

            if (!File.Exists(fileFullPath))

                return null;

            FileStream fsEncrypted =

                  new FileStream(fileFullPath, FileMode.Open, FileAccess.Read);

            try

            {

                DESCryptoServiceProvider des = new DESCryptoServiceProvider();

                des.Key = ASCIIEncoding.ASCII.GetBytes(eKey);

                des.IV = ASCIIEncoding.ASCII.GetBytes(eKey);

                CryptoStream cryptostream =

                     new CryptoStream(fsEncrypted, des.CreateDecryptor(), CryptoStreamMode.Read);

                object data = null;

                BinaryFormatter bin =

                     new BinaryFormatter(null, new StreamingContext(StreamingContextStates.File));

                using (StreamReader reader = new StreamReader(cryptostream))

                {

                    data = bin.Deserialize(reader.BaseStream);

                    reader.Close();

                }

                return data;

            }

            finally

            {

                if (fsEncrypted != null)

                    fsEncrypted.Close();

            }

        }

    }

The samples I provide here are from a Utility library that I use in some of my apps, you can download the working src and tests here.  If anyone out there finds some use in these or can suggest improvements or other applications, I’d love to hear about it.  Thanks and happy baking..

Object Bakery – Fun with .NET Serialization and Crypto: Part 1

Being able to snapshot an object graph at runtime can be a valuable capability.  I have had occasion to capture objects during a run for the purpose of analyzing and debugging problems that require comparison across multiple runs.  Additionally, the power to dehydrate objects and store them in a disk based cache, can also prove quite useful in certain scenarios.  Some other capabilities include capturing archetypal object instances to file and adding them as embedded resources to test projects to serve as  stubs and/or test doubles.  Some other ways to work with objects offline might include sending dehydrated objects through FTP, Email , Message Queues, etc.  I present here a few helper classes that I have used for the purposes described and more. 

First, when security is not a concern, serialization to plain text Xml can be an effective means of quick object dehydration.  I wrote the SerializationHelper for this purpose.  Below is a test that demonstrates instancing a serializable entity class (a Product), converting it to an Xml string representation, and converting the string back into an object.

        [TestMethod]

        public void CanSerializeToandFromXml()

        {

Product product1 = new TestProductFactory().Build()[0];

            SerializationHelper serializationHelper = new SerializationHelper();

            string sProduct1 = serializationHelper.XmlSerialize(product1);

            Product product2 = serializationHelper.XmlDeserialize<Product>(sProduct1);

            XmlDocument testLoadDoc = new XmlDocument();

            testLoadDoc.LoadXml(sProduct1); //throws exception if not well formed

 

            Assert.IsTrue(product1.ID.Equals(product2.ID) &&

                    product1.Name.Equals(product2.Name));

 

            //the products themselves are not the same instance, but instead deep copies

            Assert.IsFalse(product1.Equals(product2));

        }

As you can see the assertions, show that the two Product objects have the same data, but they are in fact not the same object.  With this we have created a deep copy of the original product as it makes it through the round trip.  The dehydrated Product looks in this case looks something like this:

<?xml version="1.0"?>

<Product xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">

  <ID>1</ID>

  <SKU>fa8f0777-74e7-4aeb-926f-a554b5dc191e</SKU>

  <Name>Monkey</Name>

  <Description>Squirrel Monkey.  Aggressive disposition.  Likes icecream and hugs.</Description>

  <Assets>

    <Asset>

      <Format>Docx</Format>

      <Data>

        UEsDBBQABgAIAAAAIQAeGe91cwEAAFQFAAATAAgCW0NvbnRlbnRfVHlwZXNdLnhtbCCiBAIooAACAAAAAAAAAAAAAAA</Data>

        <ID>1</ID>

      <Name>Brochure</Name>

    </Asset>

    <Asset>

      <Format>Jpg</Format>

      <Data>/9j/4U5uRXhpZgAATU0AKgAAAAgACgEPAAIAAAAGAAAAhgEQAAIAAAAPAAAAjAESAAMAAAABAAEAAAEaAAUAAAA</Data>

        <ID>2</ID>

      <Name>Image1</Name>

    </Asset>

    <Asset>

      <Format>Jpg</Format>

      <Data>/9j/4VsfRXhpZgAATU0AKgAAAAgACgEPAAIAAAAGAAAAhgEQAAIAAAAPAAAAjAESAAMAAAABAAEAAAEaAAUAAAAB</Data>

      <ID>3</ID>

      <Name>Image2</Name>

    </Asset>

  </Assets>

</Product>

Note that in this case, the Product class carries with it a generic List of Assets, which themselves carry byte arrays that represent images, documents, brochures, etc..  Thus with this approach I’m able to flatten an object graph that carries file assets send it somewhere as Xml, reconstitute it, save off its byte arrays to disk, and then open them with the appropriate program.

If the entity objects that you are working with do not implement ISerializable, they should be tagged with the Serializable attribute.  In the case where you have not explicitly implemented ISerializable and some of the properties are not set, you’ll find that these properties are missing from the Xml that results from serialization with XmlSerializer, SoapFormatter, or BinaryFormatter.  The DataContractSerialize/DataContractDeserialize helper methods allow you to provide a Type that is attributed with WCF DataContract and DataMember attributes to give the serializer hints on how to work with the object. 

        [TestMethod]

        public void CanDataContractSerializeDeserialize()

        {

            Product product = new TestProductFactory().Build()[0];

            var serializationHelper = new  SerializationHelper();

 

            var pBytes = serializationHelper.DataConractSerialize<Product>(product);

            string xstring = Encoding.UTF8.GetString(pBytes);

 

            var pBytes2 = Encoding.UTF8.GetBytes(xstring);

            Product product2 = serializationHelper.DataConractDeserialize<Product>(pBytes2);

 

            //verify

            Assert.IsTrue(product.Name.Equals(product2.Name) && product.ID.Equals(product2.ID));

            Assert.IsFalse(product.Equals(product2)); //deep copy

        }

Now the Class..

    public class SerializationHelper

    {

        public string XmlSerialize(object Member)

        {

            using (MemoryStream b = new MemoryStream())

            {

                XmlSerializer xs = new

                    XmlSerializer(Member.GetType());

                xs.Serialize(b, Member);

                return Encoding.UTF8.GetString(b.ToArray());

            }

        }

        public T XmlDeserialize<T>(string Serialized)

        {

            using (MemoryStream b =

                new MemoryStream(Encoding.UTF8.GetBytes(Serialized)))

                {

                    XmlSerializer xs = new XmlSerializer(typeof(T));

                    return (T)xs.Deserialize(b);

                }

        }

        public byte[] SoapSerialize<T>(T obj)

        {

            byte[] bytes = null;

            using (MemoryStream b = new MemoryStream())

            {

                SoapFormatter formatter = new SoapFormatter();

                formatter.Serialize(b, obj);

                bytes = b.ToArray();

            }

            return bytes;

        }

        public T SoapDeserialize<T>(byte[] blob)

        {

            T obj = default(T);

            using (MemoryStream b = new MemoryStream(blob))

            {

                SoapFormatter formatter = new SoapFormatter();

                obj = (T)formatter.Deserialize(b);

            }

            return obj;

        }

        public byte[] BinarySerialize(object obj)

        {

            byte[] bytes = null;

            IFormatter formatter = new BinaryFormatter();

            using (MemoryStream stream = new MemoryStream())

            {

                formatter.Serialize(stream, obj);

                bytes = stream.ToArray();

            }

            return bytes;

        }

        public T BinaryDeserialize<T>(byte[] bytes)

        {

            T instance = default(T);

            IFormatter formatter = new BinaryFormatter();

            using (MemoryStream stream = new MemoryStream(bytes))

            {

                instance = (T)formatter.Deserialize(stream);

                bytes = stream.ToArray();

            }

            return instance;

        }

        public virtual T DataConractDeserialize<T>(byte[] bytes)

        {

            T result = default(T);

            using (var stream = new MemoryStream(bytes))

            {

                XmlReader reader = XmlReader.Create(stream);

                DataContractSerializer serializer = new DataContractSerializer(typeof(T));

                result = (T)serializer.ReadObject(reader);

            }

            return result;

        }

        public virtual byte[] DataConractSerialize<T>(T obj)

        {

            byte[] bytes = new byte[] { };

            using (var stream = new MemoryStream())

            {

                XmlWriter writer = XmlWriter.Create(stream);

                DataContractSerializer serializer = new DataContractSerializer(typeof(T));

                serializer.WriteObject(writer, obj);

                writer.Flush();

                bytes = stream.ToArray();

            }

            return bytes;

        }

    }

Thats the basic scenario.  In part 2 of this article, I’ll demonstrate how to do secure object persistence to disk.  You’ll also be able to download the source for both parts.  Enjoy and happy baking…