Archive for the ‘csharp’ Category.

How to mock an Entity Framework DbContext and its DbSet properties

Entity Framework (EF) is a data access layer (DAL) that allows for easily accessing a database for your create, read, update, and delete (CRUD) actions. If you use Entity Framework, you don’t need to test your DAL CRUD actions. You just need to test your code that uses it.

With EF, you can mock your database, though it isn’t exactly easy. The database is interfaced through the DbContext. Tables are interfaces through a property of type DbSet where T is an object representing data in the table. For example, DbSet would represent your users. We can use a List to mock the database table. It would also be nice if we could easily have every table mocked by default.

Ok, now that we have established that EF 6 DbContext mocking isn’t easy, let’s change that. Let’s make it easy.

Using the following two classes, you can easily mock your DbContext with a single line of code:

var mockDbContext = EntityFrameworkMockHelper.GetMockContext<MyDbContext>();

Note: While this single line of code successfully mocks your DbContext, it doesn’t add any default data. You still have to do that work yourself, but now it should be easy because you can use the very easy to use Add method on any table.

MockDbContext.Object.People.Add(new Person{ FirstName = "John", LastName = "Doe" });

Or if you have a List of sample data already created, you can add that list with AddRange.

MockDbContext.Object.People.AddRange(SamplePeople);

And here are the two classes I wrote to help with this. I must admit, I spent over three days researching this and getting this working. So hopefully, this saves you from having to do the same.

using Moq;
using System.Collections.Generic;
using System.Data.Entity;

namespace LANDesk.Licensing.WebServices.Tests.Data
{
    public class MockedDbContext<T> : Mock<T> where T : DbContext
    {
        public Dictionary<string, object> Tables
        {
            get { return _Tables ?? (_Tables = new Dictionary<string, object>()); }
        } private Dictionary<string, object> _Tables;
    }
}
using System;
using Moq;
using System.Collections.Generic;
using System.Data.Entity;
using System.Linq;
using System.Linq.Expressions;

namespace LANDesk.Licensing.WebServices.Tests.Data
{
    public static class EntityFrameworkMockHelper
    {
        /// <summary>
        /// Returns a mock of a DbContext
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <returns></returns>
        public static MockedDbContext<T> GetMockContext<T>() where T : DbContext
        {
            var instance = new MockedDbContext<T>();
            instance.MockTables();
            return instance;
        }

        /// <summary>
        /// Use this method to mock a table, which is a DbSet{T} oject, in Entity Framework.
        /// Leave the second list null if no adds or deletes are used.
        /// </summary>
        /// <typeparam name="T">The table data type</typeparam>
        /// <param name="table">A List{T} that is being use to replace a database table.</param>
        /// <returns></returns>
        public static DbSet<T> MockDbSet<T>(List<T> table) where T : class
        {
            var dbSet = new Mock<DbSet<T>>();
            dbSet.As<IQueryable<T>>().Setup(q => q.Provider).Returns(() => table.AsQueryable().Provider);
            dbSet.As<IQueryable<T>>().Setup(q => q.Expression).Returns(() => table.AsQueryable().Expression);
            dbSet.As<IQueryable<T>>().Setup(q => q.ElementType).Returns(() => table.AsQueryable().ElementType);
            dbSet.As<IQueryable<T>>().Setup(q => q.GetEnumerator()).Returns(() => table.AsQueryable().GetEnumerator());
            dbSet.Setup(set => set.Add(It.IsAny<T>())).Callback<T>(table.Add);
            dbSet.Setup(set => set.AddRange(It.IsAny<IEnumerable<T>>())).Callback<IEnumerable<T>>(table.AddRange);
            dbSet.Setup(set => set.Remove(It.IsAny<T>())).Callback<T>(t => table.Remove(t));
            dbSet.Setup(set => set.RemoveRange(It.IsAny<IEnumerable<T>>())).Callback<IEnumerable<T>>(ts =>
            {
                foreach (var t in ts) { table.Remove(t); }
            });
            return dbSet.Object;
        }

        /// <summary>
        /// Mocks all the DbSet{T} properties that represent tables in a DbContext.
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <param name="mockedContext"></param>
        public static void MockTables<T>(this MockedDbContext<T> mockedContext) where T : DbContext
        {
            Type contextType = typeof(T);
            var dbSetProperties = contextType.GetProperties().Where(prop => (prop.PropertyType.IsGenericType) && prop.PropertyType.GetGenericTypeDefinition() == typeof(DbSet<>));
            foreach (var prop in dbSetProperties)
            {
                var dbSetGenericType = prop.PropertyType.GetGenericArguments()[0];
                Type listType = typeof(List<>).MakeGenericType(dbSetGenericType);
                var listForFakeTable = Activator.CreateInstance(listType);
                var parameter = Expression.Parameter(contextType);
                var body = Expression.PropertyOrField(parameter, prop.Name);
                var lambdaExpression = Expression.Lambda<Func<T, object>>(body, parameter);
                var method = typeof(EntityFrameworkMockHelper).GetMethod("MockDbSet").MakeGenericMethod(dbSetGenericType);
                mockedContext.Setup(lambdaExpression).Returns(method.Invoke(null, new[] { listForFakeTable }));
                mockedContext.Tables.Add(prop.Name, listForFakeTable);
            }
        }
    }
}

An Xml class to linearize xml, make pretty xml, and encoding in UTF-8 or UTF-16.

UPDATE: This is now part of the EasyXml NuGet package. There is also an EasyXml.Sources NuGet package.

OK, so I had to output an Xml for something I am doing for work. I need the Xml to be pretty. I decided I wanted a C# class that would do the equivalent of what the Xml plugin in Notepad++ does. It should create Linear Xml, Pretty Xml, etc.

I found a bunch of problems when doing this in C#:

  1. How to make an Xml linearized in C#?
  2. How to make the Xml pretty, including indents and cleaned up spacing? I could get the indents, but the spacing was a problem until I solved how to linearize the Xml.
  3. How to make the Xml declaration say UTF-8? It kept saying the Xml was in UTF-16, which was accurate because the file was UTF-16. Strings in C# are always Unicode (UTF-16).
  4. How to make UTF-8 uppercase? Once I got the Xml declaration to say UTF-8, the UTF-8 text was lowercase, utf-8, instead of uppercase. Lowercase should work, but it turns out uppercase is preferred.
  5. How to output the Xml as a file in actual UTF-8 format? It is one thing to have UTF-8 in the Xml declaration, it is quite another to actually output a file byte stream in UTF-8 vs UTF-16 (Unicode). A UTF-8 file should be 1/2 the size of a UTF-16 file.

So here is my class. I hope it helps you. All the questions are answered by this class.

using System.IO;
using System.Text;
using System.Text.RegularExpressions;
using System.Xml;
using System.Xml.Linq;

namespace Rhyous.EasyXml
{
    public class Xml
    {
        public Xml(string text)
        {
            Text = text;
        }

        /// <summary>
        /// And enumeration to select UTF8 or UTF16 encoding. This is used because the defautl UTF8 
        /// and Unicode encoding types don't capitalize the UTF characters in the Xml declaration.
        /// </summary>
        public enum XmlEncoding
        {
            UTF8, // By being first it is the default
            UTF16
        };

        /// <summary>
        /// The original Xml text as is.
        /// </summary>
        public string Text { get; private set; }

        /// <summary>
        /// The Xml in a single line (no new lines or carriage returns). The data is trimmed and no more than a single space anywhere.
        /// </summary>
        public string LinearizeXml
        {
            get { return _LinearizeXml ?? (_LinearizeXml = Clean(Document, LinearizedSettings)); }
        } private string _LinearizeXml;

        /// <summary>
        /// And XDocument representation of the Xml. It uses the Linearized Xml not the original text.
        /// </summary>
        public XDocument Document
        {
            get { return _Document ?? (_Document = XDocument.Parse(GetLinearizedXml(Text))); }
        } private XDocument _Document;

        /// <summary>
        /// The Xml with each element properly indented on a separate line. The data is trimmed and no more than a single space anywhere.
        /// </summary>
        public string PrettyXml
        {
            get { return _PrettyXml ?? (_PrettyXml = Clean(Document, PrettySettings)); }
        } private string _PrettyXml;

        /// <summary>
        /// An enum that specifies whether to use UTF-8 or Unicode (UTF-16).
        /// Changing the encoding shouldn't change the original Text but pretty and linearized 
        /// versions of the Xml should change as well as the stream.
        /// </summary>
        public XmlEncoding Encoding { get; set; }

        /// <summary>
        /// A method that outputs the Xml as a stream. It outputs using the correct Encoding.
        /// It isn't enough to write encoding="UTF-8" in the Xml declaration if the output file
        /// is still UTF-16. Botht the labeling and the actually bits in the file should match.
        /// </summary>
        /// <returns>A file stream in the configured encoding.</returns>
        public Stream ToStream()
        {
            return new MemoryStream(ToByteArray());
        }

        /// <summary>
        /// This creates a byte array using the correct encoding.
        /// 
        /// Note: Naturally, UTF-8 has half as manay bytes as UTF-16, however,
        /// if UTF-8 is n bytes, UTF-16 will be 2*N+2 bytes. This is because
        /// "UTF-8" is five characters and "UTF-16" is six characters. 
        /// So a 100 byte UTF-8 file would be 202 bytes in UTF-16. 
        /// </summary>
        /// <returns>A byte[] array of the Xml string in the configured encoding.</returns>
        public byte[] ToByteArray()
        {
            return GetEncoding().GetBytes(PrettyXml ?? "");
        }

        /// <summary>
        /// A method to get the current encoding based on the Enum value.
        /// </summary>
        /// <returns>The correct Encoding.</returns>
        private Encoding GetEncoding()
        {
            switch (Encoding)
            {
                case XmlEncoding.UTF8:
                    return XmlUTF8Encoding.Instance;
                case XmlEncoding.UTF16:
                    return XmlUnicode.Instance;
                default:
                    return XmlUnicode.Instance;
            }
        }

        /// <summary>
        /// XmlWriterSettings for linearized Xml.
        /// </summary>
        private XmlWriterSettings LinearizedSettings
        {
            get
            {
                return new XmlWriterSettings
                    {
                        Encoding = GetEncoding(),
                        Indent = false,
                        NewLineOnAttributes = false
                    };
            }
        }

        /// <summary>
        /// XmlWriterSettings for Pretty Xml.
        /// </summary>
        private XmlWriterSettings PrettySettings
        {
            get
            {
                return new XmlWriterSettings
                {
                    Encoding = GetEncoding(),
                    Indent = true,
                    IndentChars = string.IsNullOrEmpty(IndentCharacters) ? "  " : IndentCharacters,
                    NewLineOnAttributes = false,
                    NewLineHandling = NewLineHandling.Replace
                };
            }
        }

        /// <summary>
        /// The characters to use for indenting Pretty Xml
        /// </summary>
        public string IndentCharacters { get; set; }

        /// <summary>
        /// The method that uses XDocument to do make clean (pretty or linearized) Xml
        /// </summary>
        /// <param name="doc">The XDcoument version of the Xml.</param>
        /// <param name="settings">The configured XmlWriterSettings.</param>
        /// <returns>A pretty Xml string.</returns>
        private string Clean(XDocument doc, XmlWriterSettings settings)
        {
            var sb = new StringBuilder();
            var stringWriter = new StringWriterWithEncoding(sb, GetEncoding());
            using (var xmlWriter = XmlWriter.Create(stringWriter, settings))
            {
                doc.Save(xmlWriter);
                xmlWriter.Flush();
                return sb.ToString();
            }
        }

        /// <summary>
        /// A method that uses Regex to linearize Xml. The regex replaces methods are used.
        /// </summary>
        /// <param name="text">The Xml text</param>
        /// <returns>Linearized Xml string.</returns>
        private string GetLinearizedXml(string text)
        {
            // Replace all white space with a single space
            var halfclean = Regex.Replace(text, @"\s+", " ", RegexOptions.Singleline);

            // Trim after >.
            var clean75 = Regex.Replace(halfclean, @">\s+", ">");

            // Trim before <
            var fullclean = Regex.Replace(clean75, @"\s+<", "<");

            return fullclean;
        }

        /// <summary>
        /// This clas allows for the Xml to be created with the Xml declaration saying UTF-8
        /// </summary>
        private sealed class StringWriterWithEncoding : StringWriter
        {
            private readonly Encoding _Encoding;

            public StringWriterWithEncoding(StringBuilder builder, Encoding encoding)
                : base(builder)
            {
                _Encoding = encoding;
            }

            public override Encoding Encoding
            {
                get { return _Encoding; }
            }
        }

        /// <summary>
        /// This class makes the UTF-8 text in the Xml declaration show up capitalized.
        /// </summary>
        private sealed class XmlUTF8Encoding : UTF8Encoding
        {
            public override string WebName
            {
                get { return base.WebName.ToUpper(); }
            }

            public override string HeaderName
            {
                get { return base.HeaderName.ToUpper(); }
            }

            public override string BodyName
            {
                get { return base.BodyName.ToUpper(); }
            }

            public static XmlUTF8Encoding Instance
            {
                get { return _XmlUTF8Encoding ?? (_XmlUTF8Encoding = new XmlUTF8Encoding()); }
            } private static XmlUTF8Encoding _XmlUTF8Encoding;
        }

        /// <summary>
        /// This class makes the UTF-16 text in the Xml declaration show up capitalized.
        /// </summary>
        private sealed class XmlUnicode : UnicodeEncoding
        {
            public override string WebName
            {
                get { return base.WebName.ToUpper(); }
            }

            public override string HeaderName
            {
                get { return base.HeaderName.ToUpper(); }
            }

            public override string BodyName
            {
                get { return base.BodyName.ToUpper(); }
            }

            public static XmlUnicode Instance
            {
                get { return _XmlUnicode ?? (_XmlUnicode = new XmlUnicode()); }
            } private static XmlUnicode _XmlUnicode;
        }
    }
}

And here are some Unit Tests.

using LANDesk.Licensing.WebServices.Model;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System.IO;
using System.Text;

namespace Rhyous.EasyXml.Tests
{
    [TestClass]
    public class XmlTests
    {
        public string LinearUtf8Xml = "<?xml version=\"1.0\" encoding=\"UTF-8\"?><Person><FirstName>John</FirstName><MiddleName>Al Leon</MiddleName><LastName>Doe</LastName></Person>";
        public string PrettyUtf8Xml =
@"<?xml version=""1.0"" encoding=""UTF-8""?>
<Person>
  <FirstName>John</FirstName>
  <MiddleName>Al Leon</MiddleName>
  <LastName>Doe</LastName>
</Person>";
        public string PrettyUtf8XmlWithTabs =
@"<?xml version=""1.0"" encoding=""UTF-8""?>
<Person>
	<FirstName>John</FirstName>
	<MiddleName>Al Leon</MiddleName>
	<LastName>Doe</LastName>
</Person>";
        public string UglyUtf8Xml =
@"<?xml version=""1.0""
encoding=""UTF-8""?>
<Person>

<FirstName>
    John
        </FirstName>

<MiddleName>
    Al
    Leon
                </MiddleName>
  <LastName>
    


Doe
        </LastName>


</Person>";

        public string LinearUtf16Xml = "<?xml version=\"1.0\" encoding=\"UTF-16\"?><Person><FirstName>John</FirstName><MiddleName>Al Leon</MiddleName><LastName>Doe</LastName></Person>";
        public string PrettyUtf16Xml =
@"<?xml version=""1.0"" encoding=""UTF-16""?>
<Person>
  <FirstName>John</FirstName>
  <MiddleName>Al Leon</MiddleName>
  <LastName>Doe</LastName>
</Person>";
        public string UglyUtf16Xml =
@"<?xml version=""1.0""
encoding=""UTF-16""?>
<Person>

<FirstName>
    John
        </FirstName>

<MiddleName>
    Al
    Leon
                </MiddleName>
  <LastName>
    


Doe
        </LastName>


</Person>";

        [TestMethod]
        public void TestMethodLinearize()
        {
            // Arrange
            Xml xml = new Xml(PrettyUtf8Xml);

            // Act
            var actual = xml.LinearizeXml;

            // Assert
            Assert.AreEqual(LinearUtf8Xml, actual);
        }

        [TestMethod]
        public void TestMethodPretty()
        {
            // Arrange
            Xml xml = new Xml(LinearUtf8Xml);

            // Act
            var actual = xml.PrettyXml;

            // Assert
            Assert.AreEqual(PrettyUtf8Xml, actual);
        }

        [TestMethod]
        public void TestMethodLinearizeUgly()
        {
            // Arrange
            Xml xml = new Xml(UglyUtf8Xml);

            // Act
            var actual = xml.LinearizeXml;

            // Assert
            Assert.AreEqual(LinearUtf8Xml, actual);
        }

        [TestMethod]
        public void TestMethodMakeUglyPretty()
        {
            // Arrange
            Xml xml = new Xml(UglyUtf8Xml);

            // Act
            var actual = xml.PrettyXml;

            // Assert
            Assert.AreEqual(PrettyUtf8Xml, actual);
        }

        [TestMethod]
        public void TestMethodLinearizeUglyUtf16()
        {
            // Arrange
            Xml xml = new Xml(UglyUtf16Xml)
            {
                Encoding = Xml.XmlEncoding.UTF16
            };

            // Act
            var actual = xml.LinearizeXml;

            // Assert
            Assert.AreEqual(LinearUtf16Xml, actual);
        }

        [TestMethod]
        public void TestMethodMakeUglyPrettyUtf16()
        {
            // Arrange
            Xml xml = new Xml(UglyUtf16Xml)
            {
                Encoding = Xml.XmlEncoding.UTF16
            };

            // Act
            var actual = xml.PrettyXml;

            // Assert
            Assert.AreEqual(PrettyUtf16Xml, actual);
        }

        [TestMethod]
        public void TestMethodStreamIsUtf8()
        {
            // Arrange
            Xml xml = new Xml(UglyUtf8Xml)
            {
                Encoding = Xml.XmlEncoding.UTF8
            };

            // Act
            var actual = xml.ToStream();
            using (var memoryStream = new MemoryStream())
            {
                actual.CopyTo(memoryStream);
                var bytes = memoryStream.ToArray();
                // Assert
                Assert.AreEqual(154, bytes.Length);
            }
        }

        [TestMethod]
        public void TestMethodStreamIsUtf16()
        {
            // Arrange
            Xml xml = new Xml(UglyUtf16Xml)
            {
                Encoding = Xml.XmlEncoding.UTF16
            };

            // Act
            var actual = xml.ToStream();
            using (var memoryStream = new MemoryStream())
            {
                actual.CopyTo(memoryStream);
                var bytes = memoryStream.ToArray();

                // Assert
                // 310 is twice the size of 154, 308, but add 2 bytes because
                // UTF-8 is 5 characters but UTF-16 is 6 characters so it is 
                // one character longer.John
                Assert.AreEqual(310, bytes.Length);
            }
        }

        [TestMethod]
        public void TestMethodPrettyWuthTabs()
        {
            // Arrange
            Xml xml = new Xml(LinearUtf8Xml)
            {
                IndentCharacters = "\t"
            };

            // Act
            var actual = xml.PrettyXml;

            // Assert
            Assert.AreEqual(PrettyUtf8XmlWithTabs, actual);
        }

        [TestMethod]
        public void TestMethodStreamUtf8IsDifferentThanStreamUtf16()
        {
            const string text = "Hello, world!";

            var utf8 = Encoding.UTF8.GetBytes(text);
            var utf16 = Encoding.Unicode.GetBytes(text);

            Assert.AreNotEqual(utf8.Length, utf16.Length);
        }
    }
}

A cryptographically random base95 string (the ascii printable characters)

using System;
using System.Security.Cryptography;
using System.Text;

namespace LANDesk.Licensing.WebServices.Business
{
    public class CryptoRandomString
    {
        public static string GetCryptoRandomBase64String(int length)
        {
            var buffer = new byte[length];
            using (var rngCryptoServiceProvider = new RNGCryptoServiceProvider())
            {
                rngCryptoServiceProvider.GetNonZeroBytes(buffer);
            }
            return Convert.ToBase64String(buffer);
        }
        
        public static string GetCryptoRandomBaseNString(int length, byte baseN)
        {
            if (length < 1)
                throw new ArgumentException("The string length must be greater than 0!");
            if (baseN < 2)
                throw new ArgumentException("The base must be 2 or greater!");

            var buffer = new byte[length];
            var builder = new StringBuilder();

            using (var rngCryptoServiceProvider = new RNGCryptoServiceProvider())
            {
                rngCryptoServiceProvider.GetBytes(buffer);
                foreach (var b in buffer)
                {
                    var tmpbuff = new byte[] { b };
                    int max = (baseN * (256 / baseN)) - 1; // minus 1 because we start at 0
                    while (tmpbuff[0] > max)
                    {
                        rngCryptoServiceProvider.GetBytes(tmpbuff);
                    }
                    var singleChar = ByteToBaseNChar(tmpbuff[0], baseN, 32); // Start at ascii 32 (space)
                    builder.Append(singleChar);
                }
            }
            return builder.ToString();
        }

        public static string GetCryptoRandomBase95String(int length)
        {
            return GetCryptoRandomBaseNString(length, 95);
        }

        public static char ByteToBaseNChar(byte b, int baseN, int asciiOffset)
        {
            return (char)(b % baseN + asciiOffset);
        }
    }
}

And here are a few tests for it. If you can think of a additional tests, please let me know.

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Text.RegularExpressions;
using LANDesk.Licensing.WebServices.Business;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace LANDesk.Licensing.WebServices.Tests.Business
{
    [TestClass]
    public class CryptoRandomStringTests
    {
        [TestMethod]
        public void TestTestThatCorrectCharacterCountIsReturned()
        {
            const int length = 100;
            var randomString = CryptoRandomString.GetCryptoRandomBase95String(length);
            Assert.AreEqual(length, randomString.Length);
        }

        [TestMethod]
        public void TestAllCharactersAreUsed()
        {
            const int length = 1000000;
            var randomString = CryptoRandomString.GetCryptoRandomBase95String(length);
            for (int i = 32; i < 126; i++)
            {
                char c = (char)i;
                Assert.IsTrue(randomString.Contains(c.ToString()));
            }
        }

        [TestMethod]
        public void TestPerformanceTenMillionCharacters()
        {
            Stopwatch watch = new Stopwatch();
            watch.Start();
            const int length = 1000000;
            var randomString = CryptoRandomString.GetCryptoRandomBase95String(length);
            watch.Stop();
            Assert.IsTrue(watch.ElapsedMilliseconds < 1000);
        } // Elapsed Milliseconds 320 (it fluxuated a few milliseconds each run) 

        [TestMethod]
        public void TestPerformanceLoop()
        {
            Stopwatch watch = new Stopwatch();
            const int length = 16;
            watch.Start();
            for (int i = 0; i < 100000; i++)
            {
                var randomString = CryptoRandomString.GetCryptoRandomBase95String(length);
            }
            watch.Stop();
            Assert.IsTrue(watch.ElapsedMilliseconds < 1000);
        }

        [TestMethod]
        public void TestDistributionInTenMillionCharacters()
        {
            const int length = 1000000;
            const int distibution = length / 95;
            int[] margins = new int[9500];
            for (int j = 0; j < 100; j++)
            {
                var randomString = CryptoRandomString.GetCryptoRandomBase95String(length);
                for (int i = 32; i < 127; i++)
                {
                    //int count = randomString.Count(c => c == i);
                    int count = CountInstancesOfChar(randomString, (char)i);
                    margins[(j * 95) + i - 32] = count;
                }
            }
            Assert.IsTrue(Math.Abs(margins.Average() - distibution) < .5);
        }

        private int CountInstancesOfChar(string str, char c)
        {
            int count = 0;
            char[] strArray = str.ToCharArray();
            int length = str.Length;
            for (int n = length - 1; n >= 0; n--)
            {
                if (strArray[n] == c)
                    count++;
            }
            return count;
        }
    }
}

Authentication Token Service for WCF Services (Part 2 – Database Authentication)

In the previous segment, Authentication Token Service for WCF Services (Part 1), we created a project that exposes an AuthenticationTokenService and a Test1Service. The object is to first authenticate using the AuthenticationTokenService. Authentication provides a token. Calls made to additional services should include the token as a header value.

We used concrete implementations of Interfaces to do our authentication, token creation, and token validation. The concrete implementations had stubbed-in code. I used interfaces because now to change this to use database authentication, I can create concrete implementation of the same interfaces. My implementation will be different but the methods and signatures should remain the same.

Download this project here: WCF BTS DB

So here is quick table that might help you visualize what is going on. We already have the interfaces, we already have the code example code. We need to write new classes that instead of using stub example code uses database code.

Interface Concrete Code Example Class Concrete Database Class
ICredentialsValidator CodeExampleCredentialsValidator DatabaseCredentialsValidator
ITokenBuilder CodeExampleTokenBuilder DatabaseTokenBuilder
ITokenValidator CodeExampleTokenValidator DatabaseTokenValidator

OK. So we have one task to create database implementation of the interfaces. However, before we do that, we have two tasks we must do first if we are going to use a database.

  1. A database (SQL Server)
  2. A data access layer (Entity Framework)

You may already have a database, in which case, skip to Step 5& – Add Entity Framework.

Step 1 – Create the database

For now, let’s keep everything in Visual Studio. So we will create the database as a file in Visual Studio.

Note: For production deployment, we will use a real database and change the connection string in the web.config to point to the real database.

  1. Right-click on App_Data and choose Add | New Item . . . | Installed > Visual C# > Data | SQL Server Database.
  2. I named this database BasicTokenDatabase.mdf.

Step 2 – Create the User table

We will create only two tables. A user table and a Token table. A user table is needed that has at least a user and a password. The user field should be unique. The password field should NOT store the password in clear text. Instead it should store a salted hash of the password. Since we are using a salt, we need a column to store the salt. If you don’t know what a salt is, read about it here: https://crackstation.net/hashing-security.htm

  1. Double-click the database in Visual Studio.
    The Data Connections widget should open with a connection to the BasicTokenDatabase.mdf.
  2. Right-click on Tables and choose Add New Table.
  3. Keep the first Id column but also make it an identity so it autoincrements.
  4. Add three columns: User, Password, and Hash.
    The table should look as follows:

    Name Data Type Allow Nulls Default
    Id int [ ]
    User nvarchar(50) [ ]
    Password nvarchar(250) [ ]
    Salt nvarchar(250) [ ]
  5. Add a Unique constraint for the User column. I do this just by adding it to the table creation code.The SQL to create the table should look like this:
    CREATE TABLE [dbo].[User] (
        [Id]       INT            IDENTITY (1, 1) NOT NULL,
        [User]     NVARCHAR (50)  NOT NULL,
        [Password] NVARCHAR (250) NOT NULL,
        [Salt]     NVARCHAR (250) NOT NULL,
        PRIMARY KEY CLUSTERED ([Id] ASC),
        CONSTRAINT [Unique_User] UNIQUE NONCLUSTERED ([User] ASC)
    );
    
  6. Click Update to create the table.
  7. Close the table designer window.

Step 3 – Create a Token table

For the purposes of our token service, we want to create a token and store it in the database. We need a table to store the token as well as some data about the token, such as create date, and which user the token belongs to, etc.

  1. Double-click the database in Visual Studio.
    The Data Connections widget should open with a connection to the BasicTokenDatabase.mdf.
  2. Right-click on Tables and choose Add New Table.
  3. Keep the first Id column but also make it an identity so it autoincrements.
  4. Add three columns: Token, UserId, CreateDateTime.
    The table should look as follows:

    Name Data Type Allow Nulls Default
    Id int [ ]
    Token nvarchar(250) [ ]
    UserId int [ ]
    CreateDate DateTime [ ]
  5. Add a foreign key constraint for the UserId column to the Id column of the User table. I do this just by adding it to the table creation code.
  6. Add a Unique constraint for the Token column. I do this just by adding it to the table creation code. The SQL to create the table should look like this:
    CREATE TABLE [dbo].[Token] (
        [Id]         INT            IDENTITY (1, 1) NOT NULL,
        [Token]      NVARCHAR (250) NOT NULL,
        [UserId]     INT            NOT NULL,
        [CreateDate] DATETIME       NOT NULL,
        PRIMARY KEY CLUSTERED ([Id] ASC),
        CONSTRAINT [Unique_Token] UNIQUE NONCLUSTERED ([Token] ASC),
        CONSTRAINT [FK_Token_ToUser] FOREIGN KEY ([UserId]) REFERENCES [dbo].[User] ([Id])
    );
    
  7. Click Update to create the table.
  8. Close the table designer window.

Step 4 – Add a default user to the database

We need a user to test. We are going to add a user as follows:

User: user1
Password: pass1
Salt: salt1

  1. Double-click the database in Visual Studio.
    The Data Connections widget should open with a connection to the BasicTokenDatabase.mdf.
  2. Right-click on SimpleTokenConnection and choose New Query.
  3. Add SQL to insert a user.
    The sql to insert the sample user is this:

    INSERT INTO [User] ([User],[Password],[Salt]) VALUES ('user1','63dc4400772b90496c831e4dc2afa4321a4c371075a21feba23300fb56b7e19c','salt1')
    

Step 5 – Add Entity Framework

  1. Right-click on the solution and choose Manage NuGet Packages for Solution.
  2. Click Online.
  3. Type “Entity” into the search.
  4. Click Install when EntityFramework comes up.
  5. You will be prompted to accept the license agreement.

Step 6 – Add a DBContext

Entity Framework has a lot of options. Because I expect you to already have a database, I am going to use Code First to an Existing database.

  1. Create a folder called Database in your project.
  2. Right-click on the Database folder and choose Add | New Item . . . | Installed > Visual C# > Data | ADO.NET Entity Data Model.
  3. Give it a name and click OK.
    I named mine SimpleTokenDbContext.
  4. Select Code First from database.
    Your BasicTokenDatabase should selected by default. If not, you have to browse to it.
  5. I named my connection in the web.config BasicTokenDbConnection and clicked next.
  6. Expand tables and expand dbo and check the User table and the Token table.
  7. Click Finish.

You should now have three new objects created:

  1. SimpleTokenDbContext.cs
  2. Token.cs
  3. User.cs

Entity Framework will allow us to use these objects when communicating with the database.

Note: I made one change to these. Because User is a table name and a column name, Entity Framework named the class object User and the property for the user column User1. That looked wierd to me, so I renamed the User1 property to Username but I left the table with and table column named User. Token and the Token property also had this issue. I changed the Token property to be Text.

[Column(&quot;User&quot;)]
[Required]
[StringLength(50)]
public string Username { get; set; }
        [Column(&quot;Token&quot;)]
        [Required]
        [StringLength(250)]
        public string Text { get; set; }

Step 7 – Implement ICredentialsValidator

  1. Create a new class called DatabaseCredentialsValidator.cs.
  2. Use Entity Framework and the Hash class to check if those credentials match what is in the User table of the database.
using System;
using System.Linq;
using WcfSimpleTokenExample.Database;
using WcfSimpleTokenExample.Interfaces;

namespace WcfSimpleTokenExample.Business
{
    public class DatabaseCredentialsValidator : ICredentialsValidator
    {
        private readonly BasicTokenDbContext _DbContext;

        public DatabaseCredentialsValidator(BasicTokenDbContext dbContext)
        {
            _DbContext = dbContext;
        }

        public bool IsValid(Model.Credentials creds)
        {
            var user = _DbContext.Users.SingleOrDefault(u =&gt; u.Username.Equals(creds.User, StringComparison.CurrentCultureIgnoreCase));
            return user != null &amp;&amp; Hash.Compare(creds.Password, user.Salt, user.Password, Hash.DefaultHashType, Hash.DefaultEncoding);
        }
    }
}

Step 8 – Implement ITokenBuilder

  1. Create a new class called DatabaseTokenBuilder.cs.
  2. Use Entity Framework to create a new token and add it to the Token table in the database.
  3. Instead of using Guid.NewGuid, which isn’t secure because it may not be cryptographically random, we will create a better random string generator using RNGCryptoServiceProvider. Se the BuildSecureToken() method below.
using System;
using System.Linq;
using System.Security.Authentication;
using System.Security.Cryptography;
using WcfSimpleTokenExample.Database;
using WcfSimpleTokenExample.Interfaces;

namespace WcfSimpleTokenExample.Business
{
    public class DatabaseTokenBuilder : ITokenBuilder
    {
        public static int TokenSize = 100;
        private readonly BasicTokenDbContext _DbContext;

        public DatabaseTokenBuilder(BasicTokenDbContext dbContext)
        {
            _DbContext = dbContext;
        }

        public string Build(Model.Credentials creds)
        {
            if (!new DatabaseCredentialsValidator(_DbContext).IsValid(creds))
            {
                throw new AuthenticationException();
            }
            var token = BuildSecureToken(TokenSize);
            var user = _DbContext.Users.SingleOrDefault(u =&gt; u.Username.Equals(creds.User, StringComparison.CurrentCultureIgnoreCase));
            _DbContext.Tokens.Add(new Token { Text = token, User = user, CreateDate = DateTime.Now });
            _DbContext.SaveChanges();
            return token;
        }

        private string BuildSecureToken(int length)
        {
            var buffer = new byte[length];
            using (var rngCryptoServiceProvider = new RNGCryptoServiceProvider())
            {
                rngCryptoServiceProvider.GetNonZeroBytes(buffer);
            }
            return Convert.ToBase64String(buffer);
        }
    }
}

Step 9 – Implement ITokenValidator

  1. Create a new class called DatabaseTokenValidator.cs.
  2. Read the token from the header data.
  3. Use Entity Framework to verify the token is valid.
using System;
using System.Linq;
using WcfSimpleTokenExample.Database;
using WcfSimpleTokenExample.Interfaces;

namespace WcfSimpleTokenExample.Business
{
    public class DatabaseTokenValidator : ITokenValidator
    {
        // Todo: Set this from a web.config appSettting value
        public static double DefaultSecondsUntilTokenExpires = 1800;

        private readonly BasicTokenDbContext _DbContext;

        public DatabaseTokenValidator(BasicTokenDbContext dbContext)
        {
            _DbContext = dbContext;
        }

        public bool IsValid(string tokentext)
        {
            var token = _DbContext.Tokens.SingleOrDefault(t =&gt; t.Text == tokentext);
            return token != null &amp;&amp; !IsExpired(token);
        }

        internal bool IsExpired(Token token)
        {
            var span = DateTime.Now - token.CreateDate;
            return span.TotalSeconds &gt; DefaultSecondsUntilTokenExpires;
        }
    }
}

Step 10 – Update the services code

Ideally we would have our services code automatically get the correct interface implementations. But for this example, we want to keep things as simple as possible.

using System.Security.Authentication;
using System.ServiceModel;
using System.ServiceModel.Activation;
using System.ServiceModel.Web;
using WcfSimpleTokenExample.Business;
using WcfSimpleTokenExample.Database;
using WcfSimpleTokenExample.Interfaces;
using WcfSimpleTokenExample.Model;

namespace WcfSimpleTokenExample.Services
{
    [ServiceContract]
    [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
    public class AuthenticationTokenService
    {
        [WebInvoke(Method = &quot;POST&quot;, RequestFormat = WebMessageFormat.Json, ResponseFormat = WebMessageFormat.Json, BodyStyle = WebMessageBodyStyle.Bare)]
        [OperationContract]
        public string Authenticate(Credentials creds)
        {
            using (var dbContext = new BasicTokenDbContext())
            {
                ICredentialsValidator validator = new DatabaseCredentialsValidator(dbContext);
                if (validator.IsValid(creds))
                    return new DatabaseTokenBuilder(dbContext).Build(creds);
                throw new InvalidCredentialException(&quot;Invalid credentials&quot;);
            }
        }
    }
}
using System.ServiceModel;
using System.ServiceModel.Activation;
using System.ServiceModel.Web;
using System.Web;
using WcfSimpleTokenExample.Business;
using WcfSimpleTokenExample.Database;
using WcfSimpleTokenExample.Interfaces;

namespace WcfSimpleTokenExample.Services
{
    [ServiceContract]
    [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
    public class Test1Service
    {
        [OperationContract]
        [WebInvoke(Method = &quot;POST&quot;, RequestFormat = WebMessageFormat.Json, ResponseFormat = WebMessageFormat.Json, BodyStyle = WebMessageBodyStyle.Bare)]
        public string Test()
        {
            var token = HttpContext.Current.Request.Headers[&quot;Token&quot;]; // This works whether aspNetCompatibilityEnabled is true of false.
            using (var dbContext = new BasicTokenDbContext())
            {
                ITokenValidator validator = new DatabaseTokenValidator(dbContext);
                return validator.IsValid(token) ? &quot;Your token worked!&quot; : &quot;Your token failed!&quot;;
            }
        }
    }
}

I didn’t make any changes to the web.config myself. However, the web.config was changed by adding Entity Framework and a database. Download the source code to see and example of it.

Testing using Postman

The steps for testing with Postman in Part 1 should still be valid for Part 2. Just remember to remove any escape characters from the returned string. For example, if a \/ is found, remove the \ as it is an escape character. If you look at the resulting token in Postman’s Pretty tab, the escape character is removed for you.

Well, by now, you should be really getting this down. Hopefully but this point, you can now take this code and implement your own Basic Token Service BTS. Hopefully you can use this where simple token authentication is needed and the bloat of an entire Secure Token Service framework is not.

Go on and read part 3 here: Authentication Token Service for WCF Services (Part 3 – Token Validation in IDispatchMessageInspector)

Authentication Token Service for WCF Services (Part 1)

I am setting out to create a thin web UI that consists of only HTML, CSS, and Javascript (HCJ) for the front end. For the back end, I have Ajax-enabled WCF services.

I have a couple of options for authentication.

Options:

  1. Authenticate with username and password every time a service is called.
  2. Store the username and password once, then store the credentials in the session or a cookie or a javascript variable and pass them every time I call a subsequent service.
  3. Authentication to one WCF service, then store a token.

Option 1 – Authenticate every time

This is not acceptable to the users. It would be a pain to type in credentials over and over again when clicking around a website.

Option 2 – Authenticate once and store credentials

This option is not acceptable because we really don’t want to be storing credentials in cookies and headers. You could alleviate the concern by hashing the password and only storing the hash, but that is still questionable. It seems this might cause the username and password to be passed around too often and eventually, your credentials will be leaked.

Option 3 – Authenticate once and store a token

This option seems the most secure. After authenticating, a token is returned to the user. The other web services can be accessed by using the token. Now the credentials are not stored. They are only passed over the network at authentication time.

Secure Token Service

This third idea is the idea around the Secure Token Service (STS). However, the STS is designed around the idea of having a 3rd party provide authentication, for example, when you login to a website using Facebook even though it isn’t a Facebook website.

STS service implementation is complex. There are entire projects built around this idea. What if you want something simpler?

Basic Token Service (BTS)

I decided that for simple authentication, there needs to be an example on the web of a Basic Token Service.

In the basic token service, there is a the idea of a single service that provides authentication. That service returns a token if authenticated, a failure otherwise. If authenticated, the front end is responsible for passing the token to any subsequent web services. This could be a header value, a cookie or a url parameter. I am going to use a header value in my project.

Here is the design.

Basic Token Service

Since this is “Basic” it should use basic code, right? It does.

The BTS Code

Download here: WCF BTS

In Visual Studio, I chose New | Project | Installed > Templates > Visual C# > WCF | WCF Service Application.

OK, so lets do some simple code. In this example, we will put everything in code. (In part 2, I will enhance the code to look to the database.)

Ajax-enabled WCF Services

Add the Authentication WCF Service first. In Visual Studio, I right-clicked on the project and chose Add | New Item … | Installed > Visual C# > Web | WCF Service (Ajax-enabled)

<%@ ServiceHost Language="C#" Debug="true" Service="WcfSimpleTokenExample.Services.AuthenticationTokenService" CodeBehind="AuthenticationTokenService.svc.cs" %>
using System.Security.Authentication;
using System.ServiceModel;
using System.ServiceModel.Activation;
using System.ServiceModel.Web;
using WcfSimpleTokenExample.Business;
using WcfSimpleTokenExample.Interfaces;
using WcfSimpleTokenExample.Model;

namespace WcfSimpleTokenExample.Services
{
    [ServiceContract]
    [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
    public class AuthenticationTokenService
    {
        [WebInvoke(Method = "POST", RequestFormat = WebMessageFormat.Json, ResponseFormat = WebMessageFormat.Json, BodyStyle = WebMessageBodyStyle.Bare)]
        [OperationContract]
        public string Authenticate(Credentials creds)
        {
            ICredentialsValidator validator = new CodeExampleCredentialsValidator();
            if (validator.IsValid(creds))
                return new CodeExampleTokenBuilder().Build(creds);
            throw new InvalidCredentialException("Invalid credentials");
        }
    }
}

A second example service:

<%@ ServiceHost Language="C#" Debug="true" Service="WcfSimpleTokenExample.Services.Test1Service" CodeBehind="Test1Service.svc.cs" %>
using System.ServiceModel;
using System.ServiceModel.Activation;
using System.ServiceModel.Web;
using System.Web;
using WcfSimpleTokenExample.Business;
using WcfSimpleTokenExample.Interfaces;

namespace WcfSimpleTokenExample.Services
{
    [ServiceContract]
    [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
    public class Test1Service
    {
        [OperationContract]
        [WebInvoke(Method = "POST", RequestFormat = WebMessageFormat.Json, ResponseFormat = WebMessageFormat.Json, BodyStyle = WebMessageBodyStyle.Bare)]
        public string Test()
        {
            var token = HttpContext.Current.Request.Headers["Token"];
            ITokenValidator validator = new CodeExampleTokenValidator();
            if (validator.IsValid(token))
            {
                return "Your token worked!";
            }
            else
            {
                return "Your token failed!";
            }
        }
    }
}
<?xml version="1.0"?>
<configuration>
  <appSettings>
    <add key="aspnet:UseTaskFriendlySynchronizationContext" value="true" />
  </appSettings>
  <system.web>
    <compilation debug="true" targetFramework="4.5" />
    <httpRuntime targetFramework="4.5"/>
  </system.web>
  <system.serviceModel>
    <services>
      <service name="WcfSimpleTokenExample.Services.AuthenticationTokenService" behaviorConfiguration="ServiceBehaviorHttp" >
        <endpoint address="" behaviorConfiguration="AjaxEnabledBehavior" binding="webHttpBinding" contract="WcfSimpleTokenExample.Services.AuthenticationTokenService" />
      </service>
      <service name="WcfSimpleTokenExample.Services.Test1Service" behaviorConfiguration="ServiceBehaviorHttp" >
        <endpoint address="" behaviorConfiguration="AjaxEnabledBehavior" binding="webHttpBinding" contract="WcfSimpleTokenExample.Services.Test1Service" />
      </service>
    </services>
    <behaviors>
      <endpointBehaviors>
        <behavior name="AjaxEnabledBehavior">
          <webHttp helpEnabled="true" />
        </behavior>
      </endpointBehaviors>
      <serviceBehaviors>
        <behavior name="ServiceBehaviorHttp">
          <serviceMetadata httpGetEnabled="true" httpsGetEnabled="true" />
          <serviceDebug includeExceptionDetailInFaults="true" />
        </behavior>
      </serviceBehaviors>
    </behaviors>
    <serviceHostingEnvironment aspNetCompatibilityEnabled="true" multipleSiteBindingsEnabled="true" />
  </system.serviceModel>
  <system.webServer>
    <modules runAllManagedModulesForAllRequests="true"/>
    <directoryBrowse enabled="true"/>
  </system.webServer>
</configuration>

Note: In the project, there is an xdt:Transform for the web.config.debug and the web.config.release if you use web deploy. These enforce that the web services that make them only use HTTPS. Check them out.

Models

Now we are going to have a single class in the Model for this basic example, a Credentials class.

namespace WcfSimpleTokenExample.Model
{
    public class Credentials
    {
        public string User { get; set; }
        public string Password { get; set; }
    }
}

Interfaces

using WcfSimpleTokenExample.Model;

namespace WcfSimpleTokenExample.Interfaces
{
    public interface ICredentialsValidator
    {
        bool IsValid(Credentials creds);
    }
}
using WcfSimpleTokenExample.Model;

namespace WcfSimpleTokenExample.Interfaces
{
    interface ITokenBuilder
    {
        string Build(Credentials creds);
    }
}
namespace WcfSimpleTokenExample.Interfaces
{
    public interface ITokenValidator
    {
        bool IsValid(string token);
    }
}

Business Implementations

using WcfSimpleTokenExample.Interfaces;
using WcfSimpleTokenExample.Model;

namespace WcfSimpleTokenExample.Business
{
    public class CodeExampleCredentialsValidator : ICredentialsValidator
    {
        public bool IsValid(Credentials creds)
        {
            // Check for valid creds here
            // I compare using hashes only for example purposes
            if (creds.User == "user1" && Hash.Get(creds.Password, Hash.HashType.SHA256) == Hash.Get("pass1", Hash.HashType.SHA256))
                return true;
            return false;
        }
    }
}
using System.Security.Authentication;
using WcfSimpleTokenExample.Interfaces;
using WcfSimpleTokenExample.Model;

namespace WcfSimpleTokenExample.Business
{
    public class CodeExampleTokenBuilder : ITokenBuilder
    {
        internal static string StaticToken = "{B709CE08-D2DE-4201-962B-3BBAC74C5952}";

        public string Build(Credentials creds)
        {
            if (new CodeExampleCredentialsValidator().IsValid(creds))
                return StaticToken;
            throw new AuthenticationException();
        }
    }
}
using WcfSimpleTokenExample.Interfaces;

namespace WcfSimpleTokenExample.Business
{
    public class CodeExampleTokenValidator : ITokenValidator
    {
        public bool IsValid(string token)
        {
            return CodeExampleTokenBuilder.StaticToken == token;
        }
    }
}

I also use the Hash.cs file from this post: A C# class to easily create an md5, sha1, sha256, or sha512 hash.

Demo

I use the Postman plugin for Chrome.
Postman

Step 1 – Authenticate and acquire token

  1. Set the url. In this example, it is a local debug url:
    http://localhost:49911/Services/AuthenticationTokenService.svc/Authenticate.
  2. Set a header value: Content-Type: application/json.
  3. Add the body: {“User”:”user1″,”Password”:”pass1″}
  4. Click Send.
  5. Copy the GUID returned for the next step.

PostmanAuthReceive

Step 2 – Call subsequent service

  1. Set the url. In this example, it is a local debug url:
    http://localhost:49911/Services/Test1Service.svc/Test
  2. Add a header called “Token” and paste in the value received from the authentication step

PostmanTestReceive

Part 1 uses examples that are in subbed in statically in the code. In Authentication Token Service for WCF Services (Part 2 – Database Authentication), we will enhance this to use a database for credentials validation and token storage and token validation.

A C# class to easily create an md5, sha1, sha256, or sha512 hash

I started making a class to make hashing easier in C#. I found someone who had done this already here: http://techlicity.com/blog/dotnet-hash-algorithms

That was a good start. I just needed to:

  1. Add unit tests
  2. Remove the duplicate code (condense the class)
  3. Add a default values, such as a default Encoding and to use sha256 by default.
  4. Add easier salt handling.

I condensed the code (by more than half the lines by the way) and now have this much easier to read and test class:

using System.Linq;
using System.Security.Cryptography;
using System.Text;

namespace WcfSimpleTokenExample.Business
{
    public class Hash
    {
        public static Encoding DefaultEncoding = Encoding.UTF8;
        public const HashType DefaultHashType = HashType.SHA256;

        public enum HashType
        {
            MD5,
            SHA1,
            SHA256,
            SHA512
        }

        public static string Get(string text, HashType hashType = DefaultHashType, Encoding encoding = null)
        {
            switch (hashType)
            {
                case HashType.MD5:
                    return Get(text, new MD5CryptoServiceProvider(), encoding);
                case HashType.SHA1:
                    return Get(text, new SHA1Managed(), encoding);
                case HashType.SHA256:
                    return Get(text, new SHA256Managed(), encoding);
                case HashType.SHA512:
                    return Get(text, new SHA512Managed(), encoding);
                default:
                    throw new CryptographicException("Invalid hash alrgorithm.");
            }
        }

        public static string Get(string text, string salt, HashType hashType = DefaultHashType, Encoding encoding = null)
        {
            return Get(text + salt, hashType, encoding);
        }

        public static string Get(string text, HashAlgorithm algorithm, Encoding encoding = null)
        {
            byte[] message = (encoding == null) ? DefaultEncoding.GetBytes(text) : encoding.GetBytes(text);
            byte[] hashValue = algorithm.ComputeHash(message);
            return hashValue.Aggregate(string.Empty, (current, x) => current + string.Format("{0:x2}", x));
        }

        public static bool Compare(string original, string hashString, HashType hashType = DefaultHashType, Encoding encoding = null)
        {
            string originalHash = Get(original, hashType, encoding);
            return (originalHash == hashString);
        }

        public static bool Compare(string original, string salt, string hashString, HashType hashType = DefaultHashType, Encoding encoding = null)
        {
            return Compare(original + salt, hashString, hashType, encoding);
        }
    }
}

Here are some unit tests using MSTest.

using System;
using System.Text;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using WcfSimpleTokenExample.Business;

namespace WcfSimpleTokenExample.Tests
{
    [TestClass]
    public class UnitTest1
    {
        [TestMethod]
        public void MD5Test()
        {
            // Arrange
            const string tobehashed = "1";
            // Got comparison hash from here: http://www.xorbin.com/tools/md5-hash-calculator
            // Also verified with md5sums.exe
            const string expectedHash = "c4ca4238a0b923820dcc509a6f75849b";

            // Act
            string actualHash = Hash.Get(tobehashed, Hash.HashType.MD5);

            // Assert
            Assert.AreEqual(expectedHash, actualHash);
        }

        [TestMethod]
        public void Sha1Test()
        {
            // Arrange
            const string tobehashed = "1";
            // Got comparison hash from here: http://www.xorbin.com/tools/sha1-hash-calculator
            // Also verified with sha1sums.exe
            const string expectedHash = "356a192b7913b04c54574d18c28d46e6395428ab";

            // Act
            string actualHash = Hash.Get(tobehashed, Hash.HashType.SHA1);

            // Assert
            Assert.AreEqual(expectedHash, actualHash);
        }

        [TestMethod]
        public void Sha256Test()
        {
            // Arrange
            const string tobehashed = "1";
            // Got comparison hash from here: http://www.xorbin.com/tools/sha256-hash-calculator
            // Also verified with sha1sums.exe
            const string expectedHash = "6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b";

            // Act
            string actualHash = Hash.Get(tobehashed, Hash.HashType.SHA256);

            // Assert
            Assert.AreEqual(expectedHash, actualHash);
        }

        [TestMethod]
        public void Sha512Test()
        {
            // Arrange
            const string tobehashed = "1";
            // Got comparison hash from here:  http://www.miniwebtool.com/sha512-hash-generator/
            const string expectedHash = "4dff4ea340f0a823f15d3f4f01ab62eae0e5da579ccb851f8db9dfe84c58b2b37b89903a740e1ee172da793a6e79d560e5f7f9bd058a12a280433ed6fa46510a";

            // Act
            string actualHash = Hash.Get(tobehashed, Hash.HashType.SHA512);

            // Assert
            Assert.AreEqual(expectedHash, actualHash);
        }
    }
}

See descriptive Enitity Validation Errors in exception message

By default, Entity Validation Errors are not described in the DbEntityValidationException that is thrown.

If you want those displayed, here is how:

Locate your DbContext object in your project. Overload the SaveChanges() message as follows:

        public override int SaveChanges()
        {            
            try
            {
                return base.SaveChanges();
            }
            catch (DbEntityValidationException eve)
            {
                var errorMsg = new StringBuilder(eve.Message + Environment.NewLine);
                foreach (var validationResult in eve.EntityValidationErrors)
                {
                    foreach (var error in validationResult.ValidationErrors)
                    {
                        errorMsg.Append(validationResult.Entry.Entity + ": ");
                        errorMsg.Append(error.ErrorMessage + Environment.NewLine);
                    }
                }
                // Use the same exception type to avoid downstream bugs with code that catches this
                throw new DbEntityValidationException(errorMsg.ToString(), eve.EntityValidationErrors, eve);
            }
        }

I can see how the decision to not include them by default is good. Doing so could be an information disclosure security issue. However, if wrapped in a WCF service where the the entities are used as the DataContract or POCO objects, then there really isn’t much information additional information provided than what is in the wsdl.

An MVC DropDownListFor that supports all attributes

So I need a DropDownListFor that allows me to create any attribute for the select or option tags. Obviously the attributes for each option tag should come from a property in an object that is provided in a collection. I also eliminate the need of converting a collection of your custom object to a collection of SelectListItem.

Remember a DropDownList in HTML looks like this:

<select>
  <option value="1">1</option>
  <option value="2">2</option>
  <option value="3">3</option>
</select>

The current DropDownListFor uses a list of SelectListItem objects to create the option tags. However, SelectListItem only supports the value attribute and the inner text (the text between the open option tag and the close option tag). Support for any attribute should be provided.

First, I create a model to hold the attribute data. It is a very small class with only five properties, but looks like a big class due to the extensive comments.

using System.Collections.Generic;

namespace LANDesk.Licensing.Wavelink.Models
{
    public class HtmlSelectListModel<T>
    {
        /// <summary>
        /// That collection of data elements from which to create a the select options.
        /// </summary>
        public IEnumerable<T> DataObjects { get; set; }

        /// <summary>
        /// Often there is an option on top called "--Select item--" that has an empty value.
        /// </summary>
        public string EmptyValueText { get; set; }

        /// <summary>
        /// A bool value that checks for whether EmptyValueText is used or not
        /// </summary>
        public bool ShouldUseEmptyValue
        {
            get { return !string.IsNullOrWhiteSpace(EmptyValueText); }
        }

        /// <summary>
        /// This will add attributes to the <select></select> html tag.
        /// The key is the attribute, the value is the value.
        /// For example:
        ///     SelectAttributes.add("id", "select-id-1");
        /// Would result in this html:
        ///     <select id="select-id-1"></select>
        /// </summary>
        public Dictionary<string, string> SelectAttributes
        {
            get { return _SelectAttributes ?? (_SelectAttributes = new Dictionary<string, string>()); }
            set { _SelectAttributes = value; }
        } private Dictionary<string, string> _SelectAttributes;

        /// <summary>
        /// This will add attributes to the <option></option> html tag in a select list.
        /// The key is the attribute, the value is the property on the object that contains the value.
        /// A list of objects will be used to create the options. Using reflection, the property with a
        /// name matching the value will be found. If the value is "Country" then a property name Country
        /// will be found using reflection. 
        /// 
        /// Note: If inner-text is used it will not be an attribute but will be the text between the
        ///       opening and closing tags.
        /// 
        /// For example:
        ///     OptionAttributes.add("value", "CountryTwoLetter");
        ///     OptionAttributes.add("innter-text", "Country");
        /// If the the list had to objects and the Country values were "United States" and "Canada",
        /// and the CountryTwoLetter values were "US" and "CA", then the result would be this html:
        ///     <option value="US">United States</option>
        ///     <option value="US">United States</option>
        /// 
        /// Html data-[name] attributes are supported here as well.
        /// </summary>
        public Dictionary<string, string> OptionAttributes
        {
            get { return _OptionAttributes ?? (_OptionAttributes = new Dictionary<string, string>()); }
            set { _OptionAttributes = value; }
        } private Dictionary<string, string> _OptionAttributes;
    }
}

Next add an extension method to HtmlHelper.

using LANDesk.Licensing.Wavelink.Models;
using System.Collections.Generic;
using System.Linq;
using System.Linq.Expressions;
using System.Reflection;
using System.Text;

namespace System.Web.Mvc
{
    public static class DrownDownListHelper
    {
        /// <summary>
        /// Adds increased attributed functionality to DropDownListFor.
        /// </summary>
        /// <typeparam name="TModel"></typeparam>
        /// <typeparam name="TProperty"></typeparam>
        /// <typeparam name="T">The type of object in a list.</typeparam>
        /// <param name="htmlHelper">The object this method attaches to.</param>
        /// <param name="expression">The object the selected value binds to.</param>
        /// <param name="listModel">The Model object for creating this list.</param>
        /// <returns>MvcHtmlString - Html for Razor pages.</returns>
        public static MvcHtmlString DropDownListWithAttributesFor<TModel, TProperty, T>(this HtmlHelper<TModel> htmlHelper, Expression<Func<TModel, TProperty>> expression, HtmlSelectListModel<T> listModel)
        {
            var dropdown = new TagBuilder("select");
            foreach (var selectAttribute in listModel.SelectAttributes)
            {
                dropdown.Attributes.Add(selectAttribute.Key, selectAttribute.Value);
            }
            string currentValue = null;
            var metadata = ModelMetadata.FromLambdaExpression(expression, htmlHelper.ViewData);
            if (metadata != null && metadata.Model != null)
                currentValue = metadata.Model.ToString();

            var optionsBuilder = new StringBuilder();
            foreach (T item in listModel.DataObjects)
            {
                BuildOptionTags(listModel, optionsBuilder, item, currentValue);
            }
            if (string.IsNullOrWhiteSpace(currentValue) && listModel.ShouldUseEmptyValue)
            {
                optionsBuilder.Insert(0, "<option value=\"\">" + listModel.EmptyValueText + "</option>");
            }
            dropdown.InnerHtml = optionsBuilder.ToString();

            return new MvcHtmlString(dropdown.ToString(TagRenderMode.Normal));
        }

        internal static void BuildOptionTags<T>(HtmlSelectListModel<T> listModel, StringBuilder optionsBuilder, T item, string selectedValue)
        {
            var optionAttributes = GetOptionAttributes(listModel, item);
            string innerText = GetOptionInnerText(optionAttributes);

            if (optionsBuilder == null) { optionsBuilder = new StringBuilder(); }
            optionsBuilder.Append("<option ");
            bool defaultValueFound = false;
            foreach (var attribute in optionAttributes)
            {
                optionsBuilder.Append(string.Format("{0}=\"{1}\" ", attribute.Key, attribute.Value));
                if (attribute.Value == selectedValue)
                {
                    optionsBuilder.Append("selected=\"selected\" ");
                    defaultValueFound = true;
                }
            }
            optionsBuilder.Append(">" + innerText + "</option>");
        }

        internal static string GetOptionInnerText(IDictionary<string, string> optionAttributes)
        {
            string innerText;
            if (optionAttributes.TryGetValue("inner-text", out innerText))
            {
                optionAttributes.Remove("inner-text");
            }
            return innerText;
        }

        internal static Dictionary<string, string> GetOptionAttributes<T>(HtmlSelectListModel<T> listModel, T item)
        {
            var properties = item.GetType().GetProperties(BindingFlags.Public | BindingFlags.Instance).Where(p => listModel.OptionAttributes.Values.Contains(p.Name));
            var optionAttributes = new Dictionary<string, string>();
            foreach (var p in properties)
            {
                var p1 = p;
                foreach (var oa in listModel.OptionAttributes.Where(oa => p1.Name == oa.Value))
                {
                    optionAttributes.Add(oa.Key, p.GetValue(item, null).ToString());
                }
            }
            return optionAttributes;
        }
    }
}

Now you can use this in an MVC razor cshtml page. You will need a using statement like this:
@using MyNamespace.Models

    <div class="form-group">
        @Html.LabelFor(model => model.Customer.State, new { @class = "control-label col-md-2", required = "required" })
        <div class="col-md-10">
            @Html.DropDownListFor2(model => model.Customer.State, 
                new HtmlSelectListModel<State>
                {
                    DataObjects = CountryManager.Instance.States,
                    EmptyValueText = "- Select a State -",
                    SelectAttributes = new Dictionary<string, string> { { "Id", "Customer_State" }, { "Name", "Customer.State" } },
                    OptionAttributes = new Dictionary<string, string>
                                        {
                                            { "value", "StateCode" }, { "inner-text", "StateName" }, { "data-country", "CountryCode" } 
                                        }
                }
            )
            @Html.ValidationMessageFor(model => model.Customer.State)
        </div>
    </div>

It will make some awesome html. It even allows for adding data-[name] attributes. I needed a data-country attribute, which is why I wrote this class.

<select id="Customer_State" name="Customer.State">
    <option value="" >- Select a State -</option>
    <option value="AB" data-country="CA">Alberta</option>
    <option value="AK" data-country="US">Alaska</option>
    <option value="AL" data-country="US">Alabama</option>
    <option value="AR" data-country="US">Arkansas</option>
    <option value="AZ" data-country="US">Arizona</option>
    <option value="BC" data-country="CA">British Columbia</option>
    <option value="CA" data-country="US">California</option>
    <option value="CO" data-country="US">Colorado</option>
    <option value="CT" data-country="US">Connecticut</option>
    <option value="DC" data-country="US">District of Columbia</option>
    <option value="DE" data-country="US">Delaware</option>
    <option value="FL" data-country="US">Florida</option>
    ...
</select>

If you have any feedback, I would love to hear it. Please comment.

Authenticating to Salesforce with Silenium in C#

It is pretty easy to authenticate to Salesforce with Silenium in C#. Here are the steps.

  1. Add the following NuGET package to your project:
    Silenium WebDriver
  2. Use the following code:
        public static void LoginToSalesforce(string username, string password)
        {
            IWebDriver driver = new FirefoxDriver();
            driver.Url = "https://test.salesforce.com";
            driver.Navigate();
            var userNameTextBox = driver.FindElement(By.Id("username"));
            userNameTextBox.SendKeys(username);
            var passwordTextBox = driver.FindElement(By.Id("password"));
            passwordTextBox.SendKeys(password);
            var loginButton = driver.FindElement(By.Id("Login"));
            loginButton.Submit();
        }

Yes it is that easy.

EntityUpdater Generic helper for Entity Framework

So I am using Entity Framework (EF) more and more and I really like it. However, I’ve come across a problem when using WCF services and EF that doesn’t look to be perfectly solved.

Let me share my use case.

I have many entities but in this case, let’s use Customer and CustomerSite. I am using that Entity as a POCO object for a WCF service as well. I have two Systems. SAP and a custom C# licensing system, which is where I am using WCF and EF.

Customer
– CustomerId
– CustomerName

Customer Site
– CustomerSiteId
– CustomerId
– SiteName
– Address1
– Address2
– State
– Country
– Zip
– Website
– SapCustomerId

Let’s say that someone changes the Address for a customer in SAP. A post is made with the new Address. I tried a couple of built-in Entity Framework methods, but I’ve been unable to figure out how the front end client can update a single property without first querying over WCF for the existing values and having the client side update the changed values. To me, this seems to be a broken process. The client has to query the current object, which results in a call that has to traverse over the wire to the WCF service, then to the database through EF, then back to WCF and back over the wire to the client. Then logic has to exist on the client to update the appropriate fields. The client now needs a lot of logic it otherwise shouldn’t need.

If the client wants to update only one field, Address1, the client should be able to send an update request that contains only CustomerSiteId and Address1 with everything else should be left blank.

However, the logic to do this work on the server side seems difficult if not impossible without some data from the client. WCF gets in the way. The object is created by the WCF service and every field exists in the object and none of the fields are marked as “updated” or not. If only one field is changed, the other fields are default values in the instantiated object. How can the server know which fields are updated?

Well, if the client is sending a change and the change is not a default value, then we can ignore updates for properties that use default values. However, what if we want to change to a default value? What if, for example, the default value of Address2 is null. What if you want to change the database value back to null?

What if I want to update all the address fields but leave the CustomerId the same?

  1. Write a separate WCF method every time there is a field that might be updated by itself.

    Yeah, this is a nightmare. It isn’t really a solution.

  2. Write a method that any WCF service can call that takes in an Entity Type, an Id and a list of property names and their corresponding values.

    While this would work, it breaks the idea of using the Poco object. The ability to use a simple entity object is nice. Breaking away from the Entity object to a generic object like this seems the wrong thing to do. It would work, though. It would need client code to convert entities to this new object type that has an Id and a property value dictionary. Also, now the code is not clear. Instead of calling UpdateCustomer(customer), I would call UpdateEntity(myObject), which just isn’t as clear or as readable.

  3. Have a single generic method on the server that any WCF service method could call. The method would loop through the entity object and if a property’s value is default or null, that value isn’t updated.

    I like this because the logic is on the server. I can have UpdateCustomer and UpdateCustomerSite and each can call this method easily. The one problem, what if I want to set the value to null? This wouldn’t work.

  4. Have my WCF methods on the WCF server that takes in an IEnumerable of changed property names, then have generic code to loop through the Entity and mark the appropriate properties as “Changed”.

    I like this too, but if I only change one property to an actual value, I would really like to avoid creating the IEnumerable.

  5. Have both methods 3 and 4 above.

    This is what I am going with.

I created this class to help me with this.

using System;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations.Schema;
using System.Data.Entity.Infrastructure;
using System.Linq;
using System.Reflection;

namespace Rhyous.Db.Common
{
    public class EntityUpdater<T> where T : class
    {
        /// <summary>
        /// Updates the value of all properties with a value other than null or the default value for the class type. 
        /// </summary>
        /// <param name="entity">The entity to update</param>
        /// <param name="entry">A DbEntityEntry<T> entry object to mark which propeties are modified.</param>
        public void Update(T entity, DbEntityEntry<T> entry)
        {
            var props = from propertyInfo in entity.GetType().GetProperties(BindingFlags.Public | BindingFlags.Instance).Where(pi => !Attribute.IsDefined(pi, typeof(NotMappedAttribute)))
                        let val = propertyInfo.GetValue(entity, null)
                        where !IsNullOrDefault(val) && !IsCollection(val)
                        select propertyInfo;
            foreach (var pi in props)
            {
                entry.Property(pi.Name).IsModified = true;
            }
        }

        /// <summary>
        /// Updates the value of any property in the updatedPropertyNames list. 
        /// </summary>
        /// <param name="entity">The entity to update</param>
        /// <param name="updatedPropertyNames">The list of properties to update.</param>
        /// <param name="entry">A DbEntityEntry<T> entry object to mark which propeties are modified.</param>
        public void Update(T entity, IEnumerable<string> updatedPropertyNames, DbEntityEntry<T> entry)
        {
            var propInfoCollection = entity.GetType().GetProperties(BindingFlags.Public | BindingFlags.Instance);
            foreach (var pi in from pi in propInfoCollection
                               from updatedPropertyName in updatedPropertyNames
                               where pi.Name.Equals(updatedPropertyName, StringComparison.CurrentCultureIgnoreCase)
                               select pi)
            {
                entry.Property(pi.Name).IsModified = true;
            }
        }

        /// <summary>
        /// Checks if any object is null or the default value for the class type. 
        /// </summary>
        /// <typeparam name="TT">The object class</typeparam>
        /// <param name="argument">The object to test for null or default.</param>
        /// <returns></returns>
        public static bool IsNullOrDefault<TT>(TT argument)
        {
            // deal with normal scenarios
            if (argument == null) return true;
            if (Equals(argument, default(TT))) return true;

            // deal with non-null nullables
            Type methodType = typeof(TT);
            if (Nullable.GetUnderlyingType(methodType) != null) return false;

            // deal with boxed value types
            Type argumentType = argument.GetType();
            if (argumentType.IsValueType && argumentType != methodType)
            {
                object obj = Activator.CreateInstance(argument.GetType());
                return obj.Equals(argument);
            }

            return false;
        }

        /// <summary>
        /// Check if an object is a collection. This is to use to ignore complex types.
        /// </summary>
        /// <typeparam name="TT">The object class</typeparam>
        /// <param name="obj">The object to test.</param>
        /// <returns></returns>
        public static bool IsCollection<TT>(TT obj)
        {
            return obj.GetType().GetInterfaces().Any(x => x.IsGenericType && x.GetGenericTypeDefinition() == typeof(ICollection<>));
        }
    }
}

Now, I can use this class in my separate WCF services as follows.

        /// <summary>
        /// Updates the value of all properties with a value other than null or the default value for the type.
        /// Any property that is not set to a value other than null or default is ignored.
        /// </summary>
        /// <param name="customerSite">The customerSite entity to update</param>
        public int UpdateSiteSimple(CustomerSite customerSite)
        {
            UpdateSite(customerSite, null);
        }

        /// <summary>
        /// Updates the value of any property in the customerSite contained in the updatedPropertyNames list. 
        /// The values of any properties not in the updatedPropertyNames list are ignored.
        /// </summary>
        /// <param name="customerSite">The customerSite entity to update</param>
        /// <param name="updatedPropertyNames">The list of properties to update.</param>
        public int UpdateSite(CustomerSite customerSite, IEnumerable<string> updatedPropertyNames)
        {
            if (customerSite == null) { throw new Exception("The customerSite cannot be null"); }

            using (var db = new MyDbContext())
            {
                db.CustomerSites.Attach(customerSite);
                var entry = db.Entry(customerSite);
                var updater = new EntityUpdater<CustomerSite>();
                if (updatedPropertyNames == null || !updatedPropertyNames.Any())
                    updater.Update(customerSite, entry);
                else
                    updater.Update(customerSite, updatedPropertyNames, entry);
                db.SaveChanges();
            }
            return customerSite.CustomerSiteId;
        }

If you like this code and find it useful please comment.

Currently, it ignores complex types and won’t update them. I could see attempting to update complex properties in the future.

Splitting sentences in C# using Stanford.NLP

So I need to break some sentences up. I have a pretty cool regex that does this, however, I want to try out Stanford.NLP for this. Let’s check it out.

  1. Create a Visual Studio C# project.
    I chose a New Console Project and named it SentenceSplitter.
  2. Right-click on the project and choose “Manage NuGet Packages.
  3. Add the Stanford.NLP.CoreNLP nuget package.
  4. Add the following code to Program.cs (This is a variation of the code provide here: http://sergey-tihon.github.io/Stanford.NLP.NET/StanfordCoreNLP.html
    using edu.stanford.nlp.ling;
    using edu.stanford.nlp.pipeline;
    using java.util;
    using System;
    using System.IO;
    using Console = System.Console;
    
    namespace SentenceSplitter
    {
        class Program
        {
            static void Main(string[] args)
            {
                // Path to the folder with models extracted from `stanford-corenlp-3.4-models.jar`
                var jarRoot = @"stanford-corenlp-3.4-models\";
    
                const string text = "I went or a run. Then I went to work. I had a good lunch meeting with a friend name John Jr. The commute home was pretty good.";
    
                // Annotation pipeline configuration
                var props = new Properties();
                props.setProperty("annotators", "tokenize, ssplit, pos, lemma, ner, parse, dcoref");
                props.setProperty("sutime.binders", "0");
    
                // We should change current directory, so StanfordCoreNLP could find all the model files automatically 
                var curDir = Environment.CurrentDirectory;
                Directory.SetCurrentDirectory(jarRoot);
                var pipeline = new StanfordCoreNLP(props);
                Directory.SetCurrentDirectory(curDir);
    
                // Annotation
                var annotation = new Annotation(text);
                pipeline.annotate(annotation);
    
                // these are all the sentences in this document
                // a CoreMap is essentially a Map that uses class objects as keys and has values with custom types
                var sentences = annotation.get(typeof(CoreAnnotations.SentencesAnnotation));
                if (sentences == null)
                {
                    return;
                }
                foreach (Annotation sentence in sentences as ArrayList)
                {
                    Console.WriteLine(sentence);
                }
            }
        }
    }
    

    Warning! If you try to run here, you will get the following exception: Unrecoverable error while loading a tagger model

    java.lang.RuntimeException was unhandled
      HResult=-2146233088
      Message=edu.stanford.nlp.io.RuntimeIOException: Unrecoverable error while loading a tagger model
      Source=stanford-corenlp-3.4
      StackTrace:
           at edu.stanford.nlp.pipeline.StanfordCoreNLP.4.create()
           at edu.stanford.nlp.pipeline.AnnotatorPool.get(String name)
           at edu.stanford.nlp.pipeline.StanfordCoreNLP.construct(Properties A_1, Boolean A_2)
           at edu.stanford.nlp.pipeline.StanfordCoreNLP..ctor(Properties props, Boolean enforceRequirements)
           at edu.stanford.nlp.pipeline.StanfordCoreNLP..ctor(Properties props)
           at SentenceSplitter.Program.Main(String[] args) in c:\Users\jbarneck\Documents\Projects\NLP\SentenceSplitter\SentenceSplitter\Program.cs:line 20
           at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
           at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
           at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
           at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
           at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
           at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
           at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
           at System.Threading.ThreadHelper.ThreadStart()
      InnerException: edu.stanford.nlp.io.RuntimeIOException
           HResult=-2146233088
           Message=Unrecoverable error while loading a tagger model
           Source=stanford-corenlp-3.4
           StackTrace:
                at edu.stanford.nlp.tagger.maxent.MaxentTagger.readModelAndInit(Properties config, String modelFileOrUrl, Boolean printLoading)
                at edu.stanford.nlp.tagger.maxent.MaxentTagger..ctor(String modelFile, Properties config, Boolean printLoading)
                at edu.stanford.nlp.tagger.maxent.MaxentTagger..ctor(String modelFile)
                at edu.stanford.nlp.pipeline.POSTaggerAnnotator.loadModel(String A_0, Boolean A_1)
                at edu.stanford.nlp.pipeline.POSTaggerAnnotator..ctor(String annotatorName, Properties props)
                at edu.stanford.nlp.pipeline.StanfordCoreNLP.4.create()
           InnerException: java.io.IOException
                HResult=-2146233088
                Message=Unable to resolve "edu/stanford/nlp/models/pos-tagger/english-left3words/english-left3words-distsim.tagger" as either class path, filename or URL
                Source=stanford-corenlp-3.4
                StackTrace:
                     at edu.stanford.nlp.io.IOUtils.getInputStreamFromURLOrClasspathOrFileSystem(String textFileOrUrl)
                     at edu.stanford.nlp.tagger.maxent.MaxentTagger.readModelAndInit(Properties config, String modelFileOrUrl, Boolean printLoading)
                InnerException: 
    
    
  5. Download the stanford-corenlp-full-3.4.x.zip file from here: http://nlp.stanford.edu/software/corenlp.shtml#Download
  6. Extract the stanford-corenlp-full-2014-6-16.x.zip.
    Note: Over time, as new versions come out, make sure the version you download matches the version of your NuGet package.
  7. Extract the stanford-corenlp-3.4-models.jar file to stanford-corenlp-3.4-models.
    I used 7zip to extract the jar file.
  8. Copy the stanford-corenlp-3.4-models folder to your Visual Studio project files.
    Note: This is one way to include the jar file in your project. Other ways might be a copy action or another good way would be to use an app.config appSetting. I chose this way because it makes all my files part of the project for this demo. I would probably use the app.config method in production.
  9. In Visual Studio, use ctrl + left click to  highlight the stanford-corenlp-3.4-models folder and all subfolders.
  10. Open Properties (Press F4), and change the namespace provider setting to false.
  11. In Visual Studio, use ctrl + left click to  highlight the files under the stanford-corenlp-3.4-models folder and all files in all subfolders.
  12. Open Properties (Press F4), and change the Build Action to Content and the Copy to Output Directory setting to Copy if newer.
  13. Run the code.

 

Note: At first I tried to just load the model file. That doesn’t work. I got an exception. I had to set the @jarpath as shown above. I needed to copy all the contents of the jar file.

Results

Notice that I through it curve ball by ending a sentence with Jr. It still figured it out.

I went or a run. Then I went to work. I had a good lunch meeting with a friend name John Jr. The commute home was pretty good.

However, I just tried this paragraph and it did NOT detect the break after the first sentence.

Exit Room A. Turn right. Go down the hall to the first door. Enter Room B.

I am pretty sure this second failure is due to the similarity in string with a legitimate first name, middle initial, last name.

Jared A. Barneck
Room A. Turn

Now the question is, how do I train it to not make such mistakes?

Aspx CheckBoxList Alternative that allows for the OnCheckedChanged event

My team had to edit an older apsx website and add a list of CheckBoxes, which are actually html <input type="checkbox"/> tags.  We assumed that CheckBoxList would be perfect for our task. We assumed wrong.

I wrote desktop apps for years and now that I am writing web apps, I am using MVC and Razor, so I missed the aspx days. However, having to work on legacy aspx applications is catching me up.

My use case for list of CheckBoxes

Really, my use case is simple. There are icons that should show up for some products. We wanted to store a map between products and the icons in the database. On the product page, we want a list of the icons with a checkbox by each list item. This would allow the person who creates new products to select which icons to show on a product download page.

  1. Get a list of option from a database and display that list as html checkboxes.
  2. Checking a checkbox should add a row to a database table.
  3. Unchecking a checkbox should remove that row from a database table.

CheckBoxList Fails

So what I needed was an OnClick event which is actually called OnCheckedChanged in an asp:Checkbox. Well, it turns out that the CheckBoxList control doesn’t support the OnCheckedChanged event per CheckBox. The CheckBoxList doesn’t have any type of OnClick method. Instead, there is an OnSelectedIndexChanged event. But OnSelectedIndexChanged doesn’t even tell you which CheckBox was clicked. If a CheckBox is checked, then in the OnSelectedIndexChanged event, the SelectedItem equals the clicked item, however, if the CheckBox is unchecked, the SelectedItem was a completely different CheckBox that is checked.

So to me, this made the CheckBoxList control not usable. So I set out to replicate a CheckBoxList in a way that supports OnCheckedChanged.

Repeater with CheckBox for ItemTemplate

I ended up using the Repeater control with an ItemTemplate containing a CheckBox. Using this method worked pretty much flawlessly. It leaves me to wonder, why was CheckBoxList created in the first place if a Repeater with a CheckBox in the template works perfectly well.

Here is my code:

<%@ Page Title="Home Page" Language="C#" MasterPageFile="~/Site.Master" AutoEventWireup="true" CodeBehind="Default.aspx.cs" Inherits="CheckBoxListExample._Default" %>

<%@ Import Namespace="CheckBoxListExample" %>
<%@ Import Namespace="CheckBoxListExample.Models" %>

<asp:Content ID="BodyContent" ContentPlaceHolderID="MainContent" runat="server">
    <div>
        <asp:Repeater ID="Repeater1" runat="server">
            <ItemTemplate>
                <asp:CheckBox ID="cb1" runat="server" AutoPostBack="true" OnCheckedChanged="RepeaterCheckBoxChanged"
                    Text="<%# ((CheckBoxViewModel)Container.DataItem).Name %>"
                    Checked="<%# ((CheckBoxViewModel)Container.DataItem).IsChecked %>" />
            </ItemTemplate>
        </asp:Repeater>
    </div>
</asp:Content>
using System;
using System.Collections.Generic;
using System.Web.UI;
using System.Web.UI.WebControls;
using CheckBoxListExample.Models;

namespace CheckBoxListExample
{
    public partial class _Default : Page
    {
        private List<CheckBoxViewModel> _ViewModels;

        protected void Page_Load(object sender, EventArgs e)
        {
            if (!IsPostBack)
            {
                var _ViewModels = new List<CheckBoxViewModel>
                {
                    new CheckBoxViewModel {Name = "Test1", IsChecked = true},
                    new CheckBoxViewModel {Name = "Test2"},
                    new CheckBoxViewModel {Name = "Test3"}
                };
                Repeater1.DataSource = _ViewModels;
                Repeater1.DataBind();
            }
        }

        protected void RepeaterCheckBoxChanged(object sender, EventArgs e)
        {
            var cb = sender as CheckBox;
            if (cb == null) return;
            if (cb.Checked)
            {
                // Insert
            }
            else
            {
                // Delete
            }
        }
    }
}
namespace CheckBoxListExample.Models
{
    public class CheckBoxViewModel
    {
        public string Name { get; set; }
        public bool IsChecked { get; set; }
    }
}

Why to use string.format() over concatenation (Real world experience)

So today I was working on reverse engineering some old code. I will strip this down and remove proprietary info but still let you see the bug. This is a post for newbies but experienced coders might get a laugh out of this as well.

Here is the error code:

var contract = CustomerId + ProductGroupId + EndDate.ToString("yy-MM");

Now, imagine these values:

CustomerId = 12345
ProductId = 4
Date = 8/7/2014

A new developer would assume that this would return the following:

12345415-08

They would be wrong. See, CustomerId and ProductGroupId are integers. So they don’t concatenate as strings, they add as integers. The real value is this:

1234915-08

12345 + 4 = 12349. This is math, not string concatenation.

How would a developer resolve this bug? There are two solutions:

  • Use ToString() and concatenation (DO NOT USE!)
  • Use string.Concat() (USE FOR SIMPLE CONCATENATION OR PERFORMANCE!)
  • Use string.Format() (USE FOR CONCATENATION WITH FORMATTING!)

Here is the another interesting fact. I know exactly when this bug was introduced into the system. This code was written by the developer using the first solution. The developer had originally added .ToString() to the end of these objects. The developer didn’t write buggy code. He wrote code that worked.

The code used to look like this:

var contract = CustomerId.ToString() + ProductGroupId.ToString() + EndDate.ToString("yy-MM");

So what happened? If a developer didn’t break this code, who did?

I’ll tell you what happened and who did it. Resharper.

Today, developers aren’t the only ones messing with our code. We use tools that are pretty awesome to do masses of code automation for us. One of these tools is Resharper. Well, Resharper detected the ToString() methods as “Redundant Code.” Oh, in this case, it wasn’t redundant code.

The only reason I found this bug is because I was rewriting this piece of code and had to reconstruct a ContractId and I began reverse engineering this code. I noticed that the first part of the ContractId was the CustomerId up until a few months ago. After that, it seemed to be slightly off. I was able to check source from back then and see the difference and see why this bug started. Sure enough, when resharper cleaned up my redundant code, it removed the ToString() methods.

However, while this is a Resharper bug, let’s not jump so fast to blaming Resharper. I’ll submit a bug to them, but are they really at fault? I say not completely. Why? Because the developer chose to use ToString() and concatenation instead of string.Format(). This was the wrong choice. Always use string.format(). This has been preached by many, but just as many don’t listen.

What would have happened if the developer had followed the best practice to use string.Format() or string.Concat?

var contract = string.Concat(CustomerId, ProductGroupId, EndDate.ToString("yy-MM"));

or

var contract = string.Format("{0}{1}{2}", CustomerId, ProductGroupId, EndDate.ToString("yy-MM");

The bug would never had occurred. With either string.Concat() or string.Format(). In those uses, the ToString() methods really would be redundant. With or without them, the code acts the same.

But surely avoiding a Resharper bug isn’t the only reason to use string.Concat or string.Format()? Of course there are more reasons. This error was due to the ambiguous + operator. Using string.Format() or string.Concat() eliminates such ambiguities. Also, with string.Format(), the developer’s intentions are much more clear. The format decided upon is explicitly included. When I type “{0}{1}{2}”, there is no question about what the intended format is. This is why I chose string.Format() for this piece of code. It is self-documenting and unambiguous.

Conclusion

Best practices are best  practices for a reason. Use them. Using string.Format() is always preferable to string concatenation.

P.S. This old project also could have alerted us to this error if anyone would have bothered to write a Unit Test. But this code was buried deep in a method that itself was hundreds of lines long. Following the 100/10 rule and testing would have alerted us when Resharper introduced this error.

Removing the namespace from the ConnectionString name in the web.config

So we are working on moving a legacy web site to MVC4.  Yes, it is my first time doing this.  I encountered an annoying issue that I thought worth blogging about.

Ok, so I have a DLL that is pretty much nothing more than a TableAdapter. (Yes, I wish this legacy project was using Entity Framework, but alas, it is not). I have a config file and the connectionstring name has a the mode horrible namespace.

  <connectionStrings>
    <add name="Company.Division.Feature.DataAccessLayer.MyConnectionString" connectionString="Data Source=MyServer;Initial Catalog=MyDatabase;Persist Security Info=True;User ID=MyUser;Password=MyPasswd" providerName="System.Data.SqlClient" />
    <add name="MyConnectionString" connectionString="Data Source=MyServer;Initial Catalog=MyDatabase;Persist Security Info=True;User ID=MyUser;Password=MyPasswd" providerName="System.Data.SqlClient" />
  </connectionStrings>

Seriously: Company.Division.Feature.DataAccessLayer.MyConnectionString.

Yes, the same connection string is listed twice, once with the namespace and once without.

Why? Well, because all throughout the code, the original authors make call this:

ConfigurationManager.ConnectionStrings["WavelinkLMConnectionString"].ConnectionString

I would like to get rid of one of these configuration settings. Obviously to me, it makes sense to get rid of the long one.

So why doesn’t my TableAdapter library project use the shorter table adapter. Well, I figured that out. Because it is in the Settings file and the Settings file looks for the value using the namespace.

Look at this Settings.Designer.cs file

//------------------------------------------------------------------------------
// <auto-generated>
//     This code was generated by a tool.
//     Runtime Version:4.0.30319.18051
//
//     Changes to this file may cause incorrect behavior and will be lost if
//     the code is regenerated.
// </auto-generated>
//------------------------------------------------------------------------------

namespace Company.Division.Feature.DataAccessLayer.Properties {
    
    
    [global::System.Runtime.CompilerServices.CompilerGeneratedAttribute()]
    [global::System.CodeDom.Compiler.GeneratedCodeAttribute("Microsoft.VisualStudio.Editors.SettingsDesigner.SettingsSingleFileGenerator", "11.0.0.0")]
    internal sealed partial class Settings : global::System.Configuration.ApplicationSettingsBase {
        
        private static Settings defaultInstance = ((Settings)(global::System.Configuration.ApplicationSettingsBase.Synchronized(new Settings())));
        
        public static Settings Default {
            get {
                return defaultInstance;
            }
        }
        
        [global::System.Configuration.ApplicationScopedSettingAttribute()]
        [global::System.Diagnostics.DebuggerNonUserCodeAttribute()]
        [global::System.Configuration.SpecialSettingAttribute(global::System.Configuration.SpecialSetting.ConnectionString)]
        [global::System.Configuration.DefaultSettingValueAttribute("Data Source=MyServer;Initial Catalog=MyDatabase;Persist Security Info=True;User ID=MyUser;Password=MyPasswd")]
        public string MyConnectionString{
            get {
                return ((string)(this["MyConnectionString"]));
            }
        }
    }
}

OK, so every single TableAdapter requires this value, so I can’t remove it. I could change it but then everytime the settings regenerated this file, I would have to recreate teh change. Not a good idea.

Hooray for partial classes! To fix this, I removed this setting and put it in a separate partial class. In fact, I learned that in the Settings UI, you can click View Code and it will create this file for you. But for me it created in the project root and not under the Properties solution folder, so I had to move it under Properties myself.

using System.Configuration;
using System.Diagnostics;

namespace Company.Division.Feature.DataAccessLayer.Properties
{
    internal sealed partial class Settings
    {
        [ApplicationScopedSetting]
        [DebuggerNonUserCode]
        [SpecialSetting(SpecialSetting.ConnectionString)]
        [global::System.Configuration.DefaultSettingValueAttribute("Data Source=MyServer;Initial Catalog=MyDatabase;Persist Security Info=True;User ID=MyUser;Password=MyPasswd")]
        public string MyConnectionString
        {
            get
            {
                if (!string.IsNullOrWhiteSpace(ConfigurationManager.ConnectionStrings["MyConnectionString"].ConnectionString))
                    this["MyConnectionString"] = ConfigurationManager.ConnectionStrings["MyConnectionString"].ConnectionString;
                return (string)this["MyConnectionString"];
            }
        }
    }
}

As you can see, I just check if the value is set in the config file without the namespace. If it is, I set the setting to that value.

And that is it! I now can make my config have a single connection string and I don’t have to include the namespace!

  <connectionStrings>
    <add name="MyConnectionString" connectionString="Data Source=MyServer;Initial Catalog=MyDatabase;Persist Security Info=True;User ID=MyUser;Password=MyPasswd" providerName="System.Data.SqlClient" />
  </connectionStrings>

How a software developer can learn communication?

Here is how a software developer can learn to communicate with their coworkers and avoid being extrovert.

How a software engineer can learn to improve communication

Above is an image so you can pin it.

Below is code so you can copy it.

namespace Communication
{
    public class Improve
    {
        public void ImproveCommunication(bool INeedToTalkToAPersonAboutWork, bool IKnowAboutThisPersonOrTheirFamily)
        {
            if (INeedToTalkToAPersonAboutWork)
            {
                Knowledge newKnowledgeAboutPerson;
                if (IKnowAboutThisPersonOrTheirFamily)
                {
                    DoCommentOnSomethingIAlreadyKnowAboutThisPersonOrTheirFamily();
                    newKnowledgeAboutPerson = Listen();
                }
                else
                {
                    AskFindOutQuestion();
                    newKnowledgeAboutPerson = Listen();
                }

                StoreInMemoryDatabase(newKnowledgeAboutPerson);

            }
            TalkToPersonAboutWork();
        }

        public void DoCommentOnSomethingIAlreadyKnowAboutThisPersonOrTheirFamily()
        {
        }

        public void AskFindOutQuestion()
        {
        }

        public Knowledge Listen()
        {
            return new Knowledge();
        }

        public void StoreInMemoryDatabase(Knowledge newKnowledge)
        {
        }

        public void TalkToPersonAboutWork()
        {
        }
    }

    public class Knowledge
    {
    }
}