Archive for the ‘Development’ Category

Create an OData Feed with VS 2008 and 2010

Tuesday, April 20th, 2010

Creating an OData Publisher

I did a simple overview of OData in a previous post and on our recent Developer Smackdown Podcast (Episode 23) Clark and I talked about the capabilities and how you can use the OData standard to publish all kinds of data without knowing exactly how your consumers will use it.  In this post I will show how you can utilize Entity Frameworks and create an OData service VERY easily.  I will do this in Visual Studio ‘08 as well as 2010 since Microsoft loves to add a little confusion to the mix and changed the names of WCF/ADO.NET Data Services and how you initially get at them as VS projects.

First of all, “ADO.NET Data Services” in .NET 3.5 SP1/VS 2008 is now “WCF Data Services” in .NET 4/VS 2010.  The original name was “Project Astoria”.  Depending on the timing, blogs and articles on the net may be using any of those names.

VS 2008/.NET 3.5 SP1 (with Data Services Update – link below):

For these service applications I installed the AdventureWorks demo database that you can get on CodePlex which can be installed in a SQL Server database.  I installed locally in SQL Express.

1. Start Visual Studio and create a WCF Service Application.  OK, so there’s more confusion.  Why isn’t there an ADO.NET Service Application. Dunno. But this is an easy way at getting the initial project in place.  We will actually delete the service that this template creates and use an ADO.NET Data Service.

image

If you look at your solution, you will see something like this:

image

2. Delete the service class and interface.  They will just get in the way otherwise.

3. Add a new item to the project.  Select Web and an “ADO.NET Data Service”.  In a moment of pure inspiration I named mine “ODataService.svc”.

image

If you look at your references, you will see that several new assemblies were added. System.Data.Entity, System.Data.Services, System.Data.Services.Client and System.ServiceModel.Web are all added.

4. The next thing is to add the database.  I did this through Entity Frameworks.  Some people like EF, some don’t.  Thing is, EF makes this exercise stupid easy so I added a new item “ADO.NET Entity Data Model”.

image

5. This will start the wizard and the next step is to “Generate from database” and click Next.

image

6. Choose your connection and create the connection string.

 image

7. Pick the tables that you want to publish and Finish.

image

System.Security gets added to the references and the EF data model also gets added.

image

8. So you have all the parts but there is code missing.  In the data service code file “YourServiceName.svc.cs” or whatever your file name is, there are some changes. First, for OData to work the service inherits from the “DataService” class and you have to pass in your data type.  In my example, the type generated from the EF wizard is “AdventureWorksLT2008Entities”.  As a result my class definition for the service is:

public class ODataService : DataService<DataEntities.AdventureWorksLT2008Entities>

9. Inside the “InitializeService” method of your new service you will need a few lines of code as well.  Especially since out of the box that method contains nothing but comments.  Unfortunately, for those of us who preach dependency injection and the use of interfaces, you have to change the method parameter from an IDataServiceConfiguration interface type into a DataServiceConfiguration class type.  It appears that the config type contains a required “DataServiceBehavior” property that the interface doesn’t know about.

10. Within the InitializeService method you add a few lines of code.  Here’s the whole thing:

using System.Data.Services;
using System.Data.Services.Common;

namespace OdataWcfServiceApplication
{
    public class ODataService : DataService<DataEntities.AdventureWorksLT2008Entities>
    {
        // This method is called only once to initialize service-wide policies.
        // DataServiceBehavior is part of the DataServiceConfiguration type - not IDataServiceConfiguration
        public static void InitializeService(DataServiceConfiguration config)
        {
            config.SetEntitySetAccessRule("*", EntitySetRights.AllRead);

            // Use this especially for debugging.  This line will respond with additional
            // info if an error is encountered
            config.UseVerboseErrors = true;

            // This line tells your service to use version 2 of the OData services. 
            // Version 2 allows you to do more such as control paging of output 
            config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;
        }
    }
}

OK, that’s it.  You can now run the application, the service will be hosted locally and the browser will pop up.  The result you get depends on the database that you use and the tables selected but it will appear something like this:

image

To test out the OData query capabilities just add your query to the end of the URL.  For example, in my case I have a list of products that I can request. At the end of the URL I add “Product?top=10”.  This says I want the listing of products but to limit it to the first 10.  There are a lot of products in the database and if I didn’t filter then I would be waiting a while till all of the products were put in a list. Nice that I can do that.

Here is the output of that query:

image

If this is what you see…

image

and you want to see the raw data being sent back by the query (and why wouldn’t you?) you need to go into IE/Tools/Internet Options/Content tab/Feeds and Web Slices/Settings button and uncheck “Turn on feed reading view”.  You may need to restart IE for that to take effect but after that you will see the raw data feed.

 

.NET 4/VS 2010

I’m just going to go over the differences here because for the most part, it is all pretty much the same.

  • The first difference is with step 3.  Because of the new names for the services, instead of creating an “ADO.NET Data Service”, create a “WCF Data Service”.
  • In step 7, there is another option in the EF Wizard.  “Pluralize or singularize generated object names” is a checkbox that is defaulted on.  This is important because if I did everything else the same, the object that I exposed in the feed is not called “Product”, it’s now “Products” since the name was pluralized by EF.  Obviously, queries must use the correct name or they don’t work.

I believe that’s it.  Not much difference between the two.  Just remember:

  • To use OData in VS2008/.NET 3.5, you need to install the Data Services Update (link below) – It is now part of .NET 4
  • As stated above, the object names may or may not be pluralized depending on the EF Wizard.  “Product” != “Products”
  • Queries are case sensitive.  “Product” != “product”.  If you are having trouble with a query take a look at the object names
  • If you can’t see the raw data in IE, change the content setting in Internet Options (described above)
  • If you publish OData, the query processing is done on your server so keep in mind how much it is accessed, how much data is being queried and the potential complexity of the queries as this relates specifically to sizing and bandwidth

 

Resources:

OData.Org

Data Services Update for .NET 3.5 SP1: update to .NET that installs the libraries that give you the OData capabilities.

Click here for Win7 and W2K8 R2.

Click here for all other OS versions.

Technorati Tags: ,,

Put on Your OData Face – Developer Smackdown Episode #23

Monday, April 19th, 2010

Go check out our latest episode where Clark and I talk about OData.

Episode #23: Intro to OData

OData – Getting Started

Friday, March 19th, 2010

Just got back from the MIX10 conference. Hanging out with by buddy Clark Sell and doin podcasts, we took in and tried to absorb as much as we could.  Of course the big news was around Windows Phone 7 Series and Silverlight 4 but there was also quite a buzz about OData.  Personally, the moniker is a little boring but who am I to judge – the capability is the big thing.  If you haven’t heard about it, it is an open protocol for working with ATOM and JSON data feeds.  Just from that description, you might be thinking “so what” – well, keep reading. This protocol isn’t just so you can publish or consume feed data, you know you can do that.  But, now you can output a customizable data feed without knowing the full intention of the consumer ahead of time.  In other words, you can provide a data feed service that can directly respond to the consumer’s requests/queries and output the feed with only the data requested in the way best suited for them to consume and you don’t need a custom set of API endpoints to do it.

OData-logo

Some OData Facts:

  • The OData spec is around the protocol, not the implementation. (BTW, already, there are libraries available for JavaScript, PHP, Java, Objective-C and .NET)
  • It is based on REST calls through HTTP so (GET, PUT, DELETE, POST, etc.) so strictly speaking, you don’t even need a client api to access OData feeds – you could do it from a browser URL/Address line.
  • Feeds can be output in ATOM as well as JSON again, from the same endpoint.
  • The protocol is for reading from AND writing data back to a data source
  • Security around the data can be enforced

Try it:

Go to http://services.odata.org/Northwind/Northwind.svc/

The OData site has provided access to the sample Northwind database. When you get there you will see all of the data that you can access and query against. 

Notice from the result set that there is a list of all the different collections of data that you can access from this single service.  You have Customers, Shippers, Products, Suppliers, Regions, Orders, etc.  All you need to do to access any one of the collections is to type the collection name in the URL.  This example will return a list of all products:

http://services.odata.org/Northwind/Northwind.svc/Products 

Note: at this point things are case-sensitive so “Products” works but “products” won’t.  

image

If you are in Internet Explorer, and don’t see the detailed data stream, you might want to go to “Tools \ Internet Options

\ Content \ Feeds and Web Slices \ Settings”.  Then, uncheck “Turn on feed reading view”.  After that, you will see the actual feed data in the browser.

Cool. But, what if I only want products with a unit price over 50.00?  Add a filter:

http://services.odata.org/Northwind/northwind.svc/Products?$filter=UnitPrice gt 50

Notice that the UnitPrice is typed and is a Decimal.  Others are 16bit integers, 32bit integers, booleans, etc.

Now, sort:

http://services.odata.org/Northwind/northwind.svc/Products?$filter=UnitPrice gt 50&$orderby=UnitPrice

Would you rather the data being sent back be in JSON format?

http://services.odata.org/Northwind/northwind.svc/Products?$filter=UnitPrice gt 50&$orderby=UnitPrice&$format=JSON

All of this is being done through a single endpoint.  Even more important, it is being done with a standard set of query parameters which means that you will know how to publish and consume (i.e., select, extract, filter, sort and format) data coming from any feed that uses the OData Protocol.  Schweeet!

Next up…Create your own publisher with WCF Data Services.

Resources and Info:

Technorati Tags: ,,,

Preorder and Save on Visual Studio 2010 Upgrade

Thursday, March 11th, 2010

Microsoft Visual Studio 2010 Professional will launch on April 12 but you can beat the rush and secure your copy today by pre-ordering at the affordable estimated retail price of $549, a saving of $250.

If you use a previous version of Visual Studio or any other development tool then you are eligible for this upgrade. Along with all the great new features in Visual Studio 2010 (see www.microsoft.com/visualstudio) Visual Studio 2010 Professional includes a 12-month MSDN Essentials subscription which gives you access to core Microsoft platforms: Windows 7 Ultimate, Windows Server 2008 R2 Enterprise, and Microsoft SQL Server 2008 R2 Datacenter.

So visit http://www.microsoft.com/visualstudio/en-us/pre-order-visual-studio-2010 to check out all the new features and sign up for this great offer.

VSTS 2010 Guidance Documents

Monday, January 18th, 2010

The VSTS Ranger group at Microsoft (of which I am happily a member) just released a ton of guidance documentation around VSTS/TFS 2010.  Check out Jeff Beehler’s blog post for more info and links to all of the content on CodePlex:

Rangers ship VS2010 readiness materials

Wanna Play with VSTS/TFS 2010 Beta2

Wednesday, January 13th, 2010

The following virtual images were just released and will give you the ability to try the new capabilities of VSTS and TFS in a safe, preinstalled environment.  You just need to spend a little time doing the downloads – each download page consists of an EXE and several RAR files that will put themselves together into Hyper-V or VPC images.

Download from Microsoft Downloads:

Visual Studio 2010 Beta 2 (Hyper-V)  Got W2K8 with Hyper-V loaded?  This one’s for you.
Visual Studio 2010 Beta 2 (Windows [7] Virtual PC) This one was created specifically to run on Win7 and VPC
Visual Studio 2010 Beta 2 (Virtual PC 2007 SP1)  This is a normal VPC ‘07 SP1 image

All passwords are: P2ssw0rd

Not sure where to start?  How bout this…

Visual Studio 2010 and .NET Framework 4 Training Kit

Topics Include:

  • C# 4.0
  • Visual Basic 10
  • F#
  • Parallel Extensions
  • Windows Communication Foundation
  • Windows Workflow
  • Windows Presentation Foundation
  • ASP.NET 4
  • Windows 7
  • Entity Framework
  • ADO.NET Data Services
  • Managed Extensibility Framework
  • Visual Studio Team System

The Training Kit is pretty extensive and works with Visual Studio 2010 Beta 2 and .NET Framework 4 Beta 2.

All this should keep you busy for a little while anyway.

Mocks vs. Stubs

Monday, January 11th, 2010

This post is a recap of the podcast that Clark Sell and I just recorded at DeveloperSmackdown.com (Musing #12 – Are You Mocking Me?!). 

Recently, on a project, I needed to describe mocks and stubs, what the difference is, how and when they are used and their benefits.  There are quite a few blog posts out there that describe the differences but many of them are confusing and I needed a way to clarify to my team members so I created the following content.

Commonalities:

Mocks and stubs are both ways to fake out methods being tested so they think they are using a real system resource.  System resources could be internal application objects, objects that are provided by the OS, framework objects, etc.  The issue with all of those objects is that they are either expensive to use, take excessive time to create and use, are very hard to set up for use within a test or are difficult to access during a test run.  Ultimately, the problem you are trying to solve is to isolate and unit test a method and you shouldn’t care about any external objects for that test.  You should be able to assume that they will do what they are supposed to under the given test conditions.  Trying to isolate a method under test when you have to access a back-end system that includes network communications and multiple database updates is just too difficult (in my opinion).  Using a mechanism during the test that can easily simulate the call to a resource is much easier to manage.  Mocks and stubs provide that mechanism.

Both mocks and stubs can be implemented through the use of frameworks.  The frameworks are usually for and are named after mocks (Moq, RhinoMock, TypeMock, NMock…) but they can be used to generate stubs.  If you don’t want to use a framework, stubs are easier to create in a roll your own scenario since they can easily consist of a set of simple method call implementations that just turn around and return data.  Mocks are more difficult to create on your own since they have to track more information around what happened during a test as compared to just returning data.

Differences:

To put it in a single sentence, mocks are about behavior and stubs are about state.  What the hell does that mean you say!?  A unit test that uses a stub is interested in being able to make a call to an object and getting data back that will allow the method under test the ability to continue with a given set of values.  Those values are designed to flex the method’s abilities and see if it succeeds in all the proper test scenarios.  So, stubs provide state (data) back to the method under test and your unit test has all the information it needs within itself to determine success or failure.

Mocks on the other hand are about behavior.  Under the current conditions, did my method pass the right values? Did it make all of the right calls and ONLY the calls that were necessary? As a demonstration to my team, I created a simple application to show how this works.  I tested method that had the requirement of NOT making any unnecessary database updates if there are no changes in the data.  It called the database, got some data back, made some calculations and then had an unqualified database update at the end of the method.  In my unit test, I made the call and checked the return value of the method under test.  Everything seemed to be ok so the test passed right?  Well, it passed but it really shouldn’t have.  When I used a mocking framework to set up the test and created an expectation that under those conditions a database update was not warranted, IOW, I put a “verify” statement at the end of the test, the test now fails which is what should happen.  It would not pass until I changed my code to validate an update before it performs one.  The best part of all of this is that I didn’t need a real connection to a database to verify my code.  Even if I had used a real database instead of a mock, taken the time to set up the test data, ran the test, cleaned up all my tracks after the test, I still wouldn’t know that my method did the unnecessary update. 

Mocks support the fact that my unit test, on it’s own, cannot make the determination that the entire test was a success.  It needs to request that information from the mock as the last step in verifying that the test was truly a success.  Hopefully, the following helps to illustrate this.

StubSteps

Here you can see the steps involved with creating and using a stub to help perform the test.  The test creates the stub, creates the object being tested, and performs dependency injection to make the method being tested use the stub.  The method under test is called which, in turn, calls the stub.  The stub returns data allowing the method under test to do its calculations and then return to the test method.  The test method is then able to evaluate success or failure on its own and the testing is done.

MockSteps

With a mock, the main steps are similar although the “Create the Mock” step is more involved – expectations of the behavior have to be asserted that tell the mock what behavior (what calls, possible parameter values, etc.) should occur.  The various frameworks provide simple interfaces to set that up.  The big difference in the execution of the test is that for the unit test to be able to determine complete success or failure, it has to ask the mock.  Method calls could have been made properly, but then again, maybe not.  Unless we have a mechanism in place that will track that we don’t know for sure.  A mock can tell us this.

Frameworks:

The best part of using a framework is that it makes the creation of mocks AND stubs easier.  If you have an object that you have to use in a test and it has 50 public methods and 20 properties in its interface (and you will need an interface), it is way easier to use a framework that will automatically create concrete calls for you as compared to you trying to create them on your own.  So, yes, this means that even if you don’t need to test behavior, you can use a mocking framework to create stubs for your tests.  You just don’t set up expectations or call verify at the end.

Clark and I have both used Moq, NMock and RhinoMock.  Each has their own benefits and shortcomings.  I don’t have a recommendation since picking one over the other is much like picking out any application – requirements should drive the decision.  I used Moq most recently and was impressed because of the ability to use lamda statements to create succinct expectations however, on a previous project, I had very good luck with RhinoMock.

So just remember: stubs is state – mocks is behavior!

Here is a zipped version of the test project code that I mentioned above.  It has a dependency on the Moq Mocking Framework so if you want to try to run the tests you will need to download version 3.1 and add the reference to the test project.

Using Rule-Based Data Driven Tests with MSTest and Excel

Sunday, December 13th, 2009

I know that there are a ton of resources (blogs, example code, etc.) out there that show how you can implement data driven testing into your test projects.  A lot of the implementation requirements are based on which testing framework that you use.  In this post I am going to show how you can use external data files (specifically, Excel spreadsheets) to power your tests within an MSTest framework environment.  Now, you can certainly use XML and CSV files to hold your data and each has their benefits especially if you are more familiar or have more experience with them.  But, they definitely have their drawbacks.  CSV files can be difficult to maintain and control when there is a lot of rows and fields.  XML can be hard to edit with all of the tags required for its structure.  Both typically mean that you could end up with a lot of files to deploy to the tests.  I have used all of the formats but when it gets down to it, I always end up using Excel as my main data store for my tests.

The benefits of Excel files include:

  • The editor is naturally part of the environment – it’s easy to enter, copy and paste data into a spreadsheet
  • Adding or deleting fields (columns) is trivial
  • Auto-generating some of the data can also be trivial (incremented dates, numerics, etc. can sometimes be created by simple click and drag)
  • Importing test data from other sources can be done simply – I have used the Data capabilities of Excel to pull in test data from several sources (DB, CSV, XML)
  • A single file can hold just about as many spreadsheets as you want so creating a central store for many, many tests is easy
  • You can utilize “conditional formatting” within Excel that gives you visual clues to data patterns within the test data (more below)
  • One of the biggest benefits is that you can build rules (calculations) into the columns that can be used to generate additional test data and more importantly, create expected results that can be used as direct input into your tests

Building rules into the spreadsheet:

The reason why the ability to build rules into the spreadsheet columns is so important is that it makes the job of generating the test data more straightforward.  You can know how your tests should run ahead of time.  You can also adapt to changes in method requirements and may not have to change your tests since the expected results will change when the rules in the spreadsheet are changed.  So, if the requirements of the methods that you are testing change, there is a good chance that by modifying the spreadsheet rules your tests will continue to run.  Obviously, if the method signature changes (adding/deleting parameters, changing parameter types, etc.) then this is not the case but the end result still could be that you won’t have to change as much code in the test. (This is where I must say: individual results may differ, prices subject to change without notification, always consult with your physician and discontinue use if a rash forms).

How to implement Excel spreadsheets into your MSTest environment:

There are a couple steps to perform to get your tests to be able to utilize external data files in your tests.  The first is to Deploy the file and the second is to set up the file as a “DataSource” to your test method.  This holds true for all data file types but I will just describe how to do it with .xls – Excel 2003 or .xlsx – Excel 2007+ spreadsheet/workbook files. 

Let’s start with an example.  Below is a sample project that I put together.  It is the same project that I will be using for another blog post comparing the difference between mocks and stubs but for now we will just be looking at data driven testing with Excel.

SolutionExplorerView

In the TestProject, I have my tests (“CustomerBusLogicTests.cs”) and my test data in the form of .xls and a .xlsx workbook files.  The data contained in each Excel file is the same.  The only difference is how you access them.

Step 1 – Deploy:

The DeployItem attribute is the most straightforward way to deploy.  You can put the DeployItem attribute right above the test method or right above the test class.  Personally, I like placing the attribute right above the class.  This way all the deployment item files are in a single easy to find list.  For the data file(s) to be deployed properly, you should indicate the path and the filename.  Here, I am using the relative path (from the solution) into the “TestProject” folder.  Note, “TestProject” is the name of the folder and not necessarily the name of the project – see “Gotchas” below.  Since I have the data files stored in a subfolder of the project, I also include that folder name.  This is all that is necessary to deploy your test data file to the test folder.

DeploymentItem

One thing to note is that you do not have to copy the file or use it within a build action to make this work.

SpreadsheetProperties

Step 2 – Set up the DataSource:

The DataSource attribute is how you tell the individual test method which file and which spreadsheet to use in the test.  It is basically a connection string to the data being read as input to the test plus some additional parameters that point to the specifics of how to read the data.  Since there are two spreadsheet file formats there coincidentally are two ways to access that data.  The following DataSource example is for reading Excel 2003 spreadsheet files:

[DataSource("System.Data.OleDb",
   "Provider=Microsoft.Jet.OLEDB.4.0;Data Source='|DataDirectory|\\CustomerBusLogicTestData.xls';Persist Security Info=False;Extended Properties='Excel 8.0'",
         "CustomerTestData$", DataAccessMethod.Sequential)]

- DataSource is the attribute
- System.Data.OleDb indicates the type of provider that will be used to read the data
- The next line starting with “Provider” is the connection string.  In this case the “Jet” data driver is going to be used.  This is the case for .xls or Excel 2003 files.  Excel 2007/.xlsx files use the “Ace” drivers (shown below).
- Data Source identifies where to find the file.  |DataDirectory| equates to the “Out” folder that is created every time the unit tests are run.  DeploymentItem places the data file there and |DataDirectory| says go there to get the file.
- CustomerTestData$ is the name of the spreadsheet within the workbook.  Actually, the name of the spreadsheet is “CustomerTestData”.  The added dollar sign “$” is appended to the name as a requirement.  The dollar sign is not included in the actual spreadsheet name.
- DataAccessMethod.Sequential indicates how the data is to be read.  Since the data access is all managed by the MSTest framework “Sequential” is the only one that makes sense.

Here is the DataSource attribute to do the same thing but uses an Excel 2007 (.xlsx) file in the testing:

[DataSource("System.Data.OleDb",
        @"Provider=Microsoft.ACE.OLEDB.12.0;Data Source='|DataDirectory|\\CustomerBusLogicTestData.xlsx';Extended Properties=&quot;Excel 12.0 Xml;HDR=YES&quot;",
            "CustomerTestData$", DataAccessMethod.Sequential)]

Step 3 – Use the data:

Once all of the attributes are in place, the data file is included and MSTest initiates the unit tests, the test method will be called once for every row of data you have in the spreadsheet.  You do not have to worry about iterating through the data yourself because MSTest manages everything, the total number of tests will coincide with the number of rows so you get a better feel for how many tests you actually run within the suite of tests.

int monthlyPurchases = int.Parse(TestContext.DataRow["MonthlyPurchases"].ToString());
CustomerStatusEnum statusEnum = (CustomerStatusEnum)int.Parse(TestContext.DataRow["Status"].ToString());

As you can see here, TestContext.DataRow is the syntax used to get at the current row.  To access the data in a particular column enter the title of the column in quotes as an index to DataRow.  Once the data field is accessed it can then be cast and/or parsed into whatever type needed for the test.

DataTable

In addition to the data used as input for the test, the expected result can also be extracted from the spreadsheet…

CustomerStatusEnum expectedStatusEnum = (CustomerStatusEnum)int.Parse(TestContext.DataRow["ExpectedStatus"].ToString());

And then used to assert the success of the method under test:

Assert.AreEqual<CustomerStatusEnum>(expectedStatusEnum, custStatusReturned.Status);

Ok, you should now be able to use Excel spreadsheets to supply data to your tests.  The following is the full test method:

[DataSource("System.Data.OleDb",

    @"Provider=Microsoft.ACE.OLEDB.12.0;Data Source='|DataDirectory|\\CustomerBusLogicTestData.xlsx';Extended Properties=&quot;Excel 12.0 Xml;HDR=YES&quot;",

    "CustomerTestData$",

    DataAccessMethod.Sequential)]

[Owner("Mark Nichols")]

[TestMethod]

public void CustBusLogic_WhenPurchasesPerMonthChangeStatusShouldIncrementToAppropriateLevelXlsData()

{

    // Arrange

    var custDataProv = new Mock<ICustomerDataProvider>();

    int custNo = 123;

    string name = "Mark Tester";

    string address = "1313 Mockingbird Lane, Anytown, USA, 00000";

    int monthlyPurchases = int.Parse(TestContext.DataRow["MonthlyPurchases"].ToString());

    CustomerStatusEnum statusEnum = (CustomerStatusEnum)int.Parse(TestContext.DataRow["Status"].ToString());

    CustomerStatusEnum expectedStatusEnum = (CustomerStatusEnum)int.Parse(TestContext.DataRow["ExpectedStatus"].ToString());

 

    var custStatus = new CustomerStatus { Name = name, Address = address, Status = statusEnum };

    var expectedCustStatus = new CustomerStatus { Name = name, Address = address, Status = expectedStatusEnum };

 

    // Set behavior expectations

    custDataProv.Setup(c => c.GetCustomerStatus(custNo))

        .Returns(custStatus);

 

    custDataProv

        .Setup(c => c.GetPurchaseCountForMonth(custNo))

        .Returns(monthlyPurchases);

 

    // This method is only called if the status actually changes

    if (statusEnum != expectedStatusEnum)

    {

        custDataProv

            .Setup(c => c.SetCustomerStatus(custStatus))

            .Returns(true);

    }

 

    // Act

    var custBusLogic = new CustomerBusLogic(custDataProv.Object, new MessageWrapper());

 

    var custStatusReturned = custBusLogic.CertifyCustomerPriorityStatus(custNo);

 

    // Assert

    Assert.IsNotNull(custStatusReturned, "CertifyCustomerPriorityStatus failed to return a customer status object");

    Assert.AreEqual<CustomerStatusEnum>(expectedStatusEnum, custStatusReturned.Status);

 

    if (statusEnum == expectedStatusEnum)

    {

        custDataProv.Verify(c => c.SetCustomerStatus(It.IsAny<CustomerStatus>()), Times.Never());

    }

 

    custDataProv.VerifyAll();

}

 

Building rules into the test data spreadsheet:

As I mentioned before one of the benefits of using a spreadsheet is the fact that you can build rules into the spreadsheet that calculate expected results.  This way your test can be as simple as possible and only needs to compare (assert) values to validate the test.  In addition, you can utilize conditional formatting to help you visualize the data and the rules instead of just looking at the data values themselves.

Here is an example of a spreadsheet full of test data from an actual project.  The data has been scrubbed but still shows the intent.  In this case I entered rules and calculations in multiple locations.  Sometimes the rules can be so extensive that it makes sense to break them up into multiple steps or “gates” which are not actually used in the test method.  Below there are 3 separate gate calculations used to ultimately generate the ExpectedResult.  I also used conditional formatting in Excel to show color in addition to the values so I could visualize data patterns. 

RuleBasedSheet 

This is the rule embedded in each cell under ExpectedResult:

=IF(J36,IF(A36="SCH",N36,IF(OR(A36="SPT",A36="OWD",A36="CTG"),FALSE,TRUE)),FALSE)

Trying to build these rules inside of a test method would make it far more complicated, more difficult to change if the business rules and requirements changed and you would spend more time on the test method trying to make sure that it worked properly given all of the possible permutations.  By the way, this sheet contained almost 600 rows of data so in the end, a single test method counts for that many individual tests.  Clearly, you can spend a significant amount of time creating the data in a spreadsheet especially when there is a lot of possible permutations but once you get the hang of it, I believe you will save time using simple copy/paste, conditional formatting, formulas and just the benefit of accessing all of your data in a centralized repository.

Gotcha’s when doing data driven testing with MSTest and Excel Spreadsheets:

  • Deploying the data file: if your project name differs from the actual folder name holding the project files, you must use the actual folder name to indicate where the data files are coming from.  I have run into a couple situations where (unbeknownst to me) the project name was changed after the project was created or a different folder name was used compared to the project name when it was created.  So, after a near concussion from pounding my head into the desk while telling myself that there was nothing wrong with my code, I realized that I was pointing to a folder that didn’t actually exist.  A quick change and couple Advil later, all was working.
  • Deleting rows of data from the spreadsheet:  unless you delete the actual row within a spreadsheet (not just the data in the cells) your tests may fail because of null values.  The data access mechanism that grabs data from a spreadsheet is reliant upon the metadata in the spreadsheet to know how many rows exist.  If you add data to a row and then delete that data from the cells, the spreadsheet won’t know that you do not want that row included.  So, highlight the rows, right-click and select ‘delete’ instead of just hitting the delete key.
  • Multiple data types in a single Excel column: sometimes as you enter data in a spreadsheet, the values seem to disappear as they are read into the test.  This happens when (for example) you enter numeric data and then string data in the same column.  Or, for whatever reason, Excel just interprets what you enter as two different types.  If this happens, highlight the column, right-click and set the data format manually so that all value types are the same.
  • Excel 2007 drivers:  You may or may not have the drivers loaded on your client or build machine to access Excel 2007 files.  Here is the link to the driver install and yes, it is called AccessDatabaseEngine.exe.  Once you install that driver your machine will be able to open and use .xlsx workbooks for data driven tests.

Here is a zipped version of the test project.  Because this app is also used to demonstrate the use of mocks and stubs it has a dependency on the Moq mocking framework so if you want to try to run the tests you will need to download version 3.1 and add the reference to the test project.

Windows 7 Power Configuration Diagnostics Report

Friday, September 18th, 2009

I came across a Windows 7 utility (“PowerCfg.exe”) that does a scan of your system (actually watches for 60 seconds) and then generates an HTML report describing what it found as errors, warnings and general information.  I gave it a go just to see what it said and was surprised when I got spanked with a series of error level messages.  Of course I thought that it was just being nit-picky but then realized that it was telling me that I made some mods to the energy profiles but never changed them back.  I did the changes because of a presentation that I was part of and didn’t want my machine going dim or even worse going to sleep. 

The report categorizes the power policies as “On Battery” or “Plugged In” and some of my On Battery settings were definitely off.  I’m sure the day was coming where I would be running unplugged and the battery would have been draining fast as I went merrily on running the laptop at full bore.  The power plan was set to “High Performance”.  I had no timeouts on the display or when to put it to sleep.  The disk was set to run full speed whether I was using it or not. 

In addition, in the warning section, there was a list of applications that were sucking up the majority of the CPU.  Not surprising, a virtual PC session was grabbing a bunch but Internet Explorer was chewing up even more.  However, as the report displayed, it was the Flash component within the page that IE was displaying that took up more than 43% of the 47% CPU utilization that the POWERCFG utility was complaining about.  Hmmm – good to know.

Ultimately, this is a good utility to run every once in a while to review the current state of your machine and then take action to reduce settings and extend battery life.  But wait, that’s not all!  I had some settings that were changed for that same presentation that make my laptop suck up more energy than it needs when it’s plugged in.  With the ice caps melting, Polar bears losing their environment and the Ozone layer disintegrating, every little bit helps.  I don’t need my display on high or my disk drive spinning when I am away from the machine for a while.  Nor do I need it to remain powered up all night.  All this wastes a bunch of energy.

polar-bear1

Now, I know you’re saying to yourself…”I wanna become a more responsible citizen too. How do I do it?”.  Well, here it is:

  1. Open up a command window with elevated privileges.  In other words, right click on cmd.exe and select “Run as Administrator”
  2. Change the current directory to a location where you can write a file and then be able to read it.  The PowerCfg.exe utility is going to generate an html file.
  3. Type the following command: powercfg –energy –output energy-report.html

Run PowerCfg

Look at that would ya? 17 errors and 24 warnings.  Seems like a lot.  You can type “energy-report.html” at the command prompt and the report will pop up in your browser.  Most of my errors were due to the “maximum performance under all conditions” setting that I chose.

After going to my laptop’s power utility I set it to adapt to the cpu requirements rather than just go all out, also set it to power things down and go to sleep when I’m not using it.  Errors dropped to 4 and fewer warnings as well.  Much better.

And here’s the proof – look how much happier they are now.

070907214013_Polar_Bears_LG

Moral of the story is: review your performance settings every once in a while – the polar bears will thank you.

Automated Language Translation With Powershell

Tuesday, August 11th, 2009

This is an addition to a previous blog where I used Powershell to update multiple language/culture text resource files within a WPF application. Here is the previous blog.

While I was working on the previous solution using Powershell to generate resource files using data from Excel it occurred to me that I may have situations where I’ve got my default language key/value pairs filled in but I may not have all of my translations completed.  What do I do about those empty cells?  I thought it would be better if the script filled in the blanks for me.  Yes, the translation may not be 100% accurate but it would allow me to test the application and get a good feeling about how the translated text would fit within the given user interface.

I still want the Excel spreadsheet to be the “single source of truth” so I don’t want to automatically update that data.  I want to keep the blanks in it so I know what still needs to be translated and verified.  The only thing I want updated is the resource files. 

To do the translation I looked up how to utilize the Bing Translator’s web services.  You can find out more on the Bing developer’s resources here.  To work with the web services you need an application ID which you can get here.  The Powershell script below uses an embedded application ID to complete the web service call.

Ok, now to the details.  In addition to the code in the previous blog, I added a “Translate-Table” function that looks at the dataset that is created from the Excel spreadsheet.  When it finds a missing value, it makes a call to the Bing Translator and passes in the value from the default language.  When it receives the value back, it inserts the value into the table.  Again, the Excel spreadsheet is not updated.

# function Translate-Table

# Scan the dataset extracted from Excel and look for cells that

#  are blank and therefore need to be translated.  When a blank

#  cell is found, call the Bing (Microsoft) Translator, pass the

#  from/to languages ID's and the terms to Bing and get the 

#  translated text back.  The translated text is then stored in the

#  dataset.

function Translate-Table {

    param ( [Data.DataSet] $excelData, $translationList )

 

    $count = [string]$translationList.Length

    

    foreach ($row in $excelData.Tables[0])

    {

        for ($index = 2; $index -le $count; $index++) {

            

            # We have to check for DBNull here to see if the term is present or not

            if ( $row[$index] -eq [DBNull]::Value ) {

                $fromText = [System.Web.HttpUtility]::UrlEncode($row[1])

                $fromLang = $translationList[0]

                $toLang = $translationList[$index - 1]

    

                [string] $requestString = "http://api.bing.net/xml.aspx?AppId=$appId&Query=$fromText&Sources=Translation&Version=2.2&Translation.SourceLanguage=$fromLang&Translation.TargetLanguage=$toLang"

        

                # Create and initialize the request.

                [System.Net.HttpWebRequest] $request = [System.Net.HttpWebRequest]::Create($requestString)

                    

                # Send the request; display the response.

                [System.Net.HttpWebResponse] $response = [System.Net.HttpWebResponse] $request.GetResponse()

            

                # Load the response into an XmlDocument.

                [System.Xml.XmlDocument] $xmlDocument = New-Object System.Xml.XmlDocument

                $xmlDocument.Load($response.GetResponseStream())

    

                # Add the default namespace to the namespace manager.

                # psbase: this means "return the original object".  This is neccessary because PS sometimes interprets methods and properties

                #  out of objects.  When this is the case, psbase gives you access to the root object and all of the original methods and props

                [System.Xml.XmlNamespaceManager] $nsmgr = New-Object System.Xml.XmlNamespaceManager $xmlDocument.psbase.NameTable

                $nsmgr.AddNamespace("api", "http://schemas.microsoft.com/LiveSearch/2008/04/XML/element")

    

                [System.Xml.XmlNodeList] $errors = $xmlDocument.DocumentElement.SelectNodes("./api:Errors/api:Error", $nsmgr)

    

                if ($errors.Count -gt 0)

                {

                    # There are errors in the response. Display error details.

                    Write-Output "Error retrieving translation data"

                }

                else

                {

                    $root = $xmlDocument.DocumentElement

                    

                    # Add the Translation SourceType namespace to the namespace manager.

                    $nsmgr.AddNamespace("tra", "http://schemas.microsoft.com/LiveSearch/2008/04/XML/translation")

            

                    [System.Xml.XmlNodeList] $results = $root.SelectNodes("./tra:Translation/tra:Results/tra:TranslationResult", $nsmgr)

            

                    $row.item($index) = $results.item(0).InnerText

                }

            }

        }

    }

    

    $excelData

}

 

A couple of additional variables are used to provide the application ID and the list of Bing language specifiers used to tell Bing what languages to translate from and to.

# These are the languages for each of the DataSet Translations - 

# This is for use within the Bing auto translation feature

# If there are cells in the Excel spreadsheet that are empty, they will be filled

#  in with a call to Bing's translation service.

$translationLanguages = @("En", "Es", "Ja")

$appId = "Insert your Bing Application ID Here"

 

Lastly, the call to “Translate-Table” is made immediately before the loop that generates the resource files.

# Perform translations using Bing Translator.  Only empty cells

#  in the non-default language columns will be translated

#  Note: This DOES NOT modify the Excel Spreadsheet.  It does the

#  translation and updates the resource files ONLY.  This step

#  is NOT required to generate resource files.  It is here to

#  "fill in the blanks" when the source spreadsheet does not 

#  have all cells filled in.  If you do not wish to use it,

#  comment the next line.

$excelData = Translate-Table $excelData $translationLanguages

 

Other than that, the script is the same as the original.  With this addition, you can build your translation table in Excel as the translations become available and still be able to generate your resource files and be able to run and test the application with “reasonable” translations available.

Here is the full script:

# ResourceGeneratorWithBing.ps1

#

# Summary: Script that will extract region-specific text data from an Excel

# spreadsheet and update resource files.  This allows you to create a master

# spreadsheet that contains all of your translated application text data in one

# central location.  The data in the resx files are then slaves to the 

# spreadsheet.  This way you have one source of the "true" text data within

# a multi-language application.

 

# NOTE: This script will not work in 64bit Powershell because of the lack of a

#  64bit OleDb driver for Excel.  It will work on (in fact was developed on) a 

#  64bit machine - you just have to run 32bit Powershell.

#

# Steps to make this work in your application are toward the bottom of this script

 

# function: Get-ExcelLanguageData

# Extract the ID and text values from the spreadsheet

# This data will be directly used to create the region

# specific resource files later

function Get-ExcelLanguageData {

    param( [string] $excelFilePath )

    

    # Use OleDb to open the Excel spreadsheet

    $selectStatement = "Select * from [$sheetName] Where TextId <> `"`""

 

    $connection = New-Object Data.OleDb.OleDbConnection "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=`"$excelFilePath`";Extended Properties=`"Excel 12.0 Xml;HDR=YES`";"

    $command = New-Object Data.OleDb.OleDbCommand $selectStatement, $connection

        

    $connection.Open()

        

    $dataAdapter = New-Object Data.OleDb.OleDbDataAdapter $command

        

    $dataSet = New-Object Data.DataSet

        

    [Void] $dataAdapter.Fill($dataSet)

        

    $connection.Close()

        

    return $dataSet

}

 

# function Create-ResxFile

# Use the region-specific language data (the dataset created from Excel) 

#  and modify the EXISTING resource files from my C# project.

function Create-ResxFile {

    param ( [string] $resxFilePath, [Data.DataSet] $excelData, [string] $languageColumn )

    

    $xmlDoc = new-object "System.Xml.XmlDocument"

 

    $xmlDoc.Load($resxFilePath)

  

    $dataNodeList = $xmlDoc.selectNodes("/root/data")

 

    # Remove any existing data nodes since they will all be replaced with the data from excel

    foreach($dataNode in $dataNodeList)

    {

        $dataNode.parentNode.removeChild($dataNode);

    }

 

    # Create each of the "data" elements in XML - this is where the 

    #  text data resides with the resource key

    foreach ($row in $excelData.Tables[0])

    {

        $dataElem = $xmlDoc.CreateElement("data")

        $nameAttr = $xmlDoc.CreateAttribute("name")

 

        $preserveAttr = $xmlDoc.CreateAttribute("xml:space")

        $preserveAttr.Value = "preserve"

 

        $nameAttr.Value = $row['TextId']

 

        $dataElem.Attributes.Append($nameAttr)

        $dataElem.Attributes.Append($preserveAttr)

        $dataElem.InnerXml = $row[$languageColumn]

 

        $rootNode = $xmlDoc.selectSingleNode("/root")

 

        $rootNode.AppendChild($dataElem);

    }

 

    # Save the resource file

    $xmlDoc.Save($resxFilePath)

}

 

 

# function Translate-Table

# Scan the dataset extracted from Excel and look for cells that

#  are blank and therefore need to be translated.  When a blank

#  cell is found, call the Bing (Microsoft) Translator, pass the

#  from/to languages ID's and the terms to Bing and get the 

#  translated text back.  The translated text is then stored in the

#  dataset.

function Translate-Table {

    param ( [Data.DataSet] $excelData, $translationList )

 

    $count = [string]$translationList.Length

    

    foreach ($row in $excelData.Tables[0])

    {

        for ($index = 2; $index -le $count; $index++) {

            

            # We have to check for DBNull here to see if the term is present or not

            if ( $row[$index] -eq [DBNull]::Value ) {

                $fromText = [System.Web.HttpUtility]::UrlEncode($row[1])

                $fromLang = $translationList[0]

                $toLang = $translationList[$index - 1]

    

                [string] $requestString = "http://api.bing.net/xml.aspx?AppId=$appId&Query=$fromText&Sources=Translation&Version=2.2&Translation.SourceLanguage=$fromLang&Translation.TargetLanguage=$toLang"

        

                # Create and initialize the request.

                [System.Net.HttpWebRequest] $request = [System.Net.HttpWebRequest]::Create($requestString)

                    

                # Send the request; display the response.

                [System.Net.HttpWebResponse] $response = [System.Net.HttpWebResponse] $request.GetResponse()

            

                # Load the response into an XmlDocument.

                [System.Xml.XmlDocument] $xmlDocument = New-Object System.Xml.XmlDocument

                $xmlDocument.Load($response.GetResponseStream())

    

                # Add the default namespace to the namespace manager.

                # psbase: this means "return the original object".  This is neccessary because PS sometimes interprets methods and properties

                #  out of objects.  When this is the case, psbase gives you access to the root object and all of the original methods and props

                [System.Xml.XmlNamespaceManager] $nsmgr = New-Object System.Xml.XmlNamespaceManager $xmlDocument.psbase.NameTable

                $nsmgr.AddNamespace("api", "http://schemas.microsoft.com/LiveSearch/2008/04/XML/element")

    

                [System.Xml.XmlNodeList] $errors = $xmlDocument.DocumentElement.SelectNodes("./api:Errors/api:Error", $nsmgr)

    

                if ($errors.Count -gt 0)

                {

                    # There are errors in the response. Display error details.

                    Write-Output "Error retrieving translation data"

                }

                else

                {

                    $root = $xmlDocument.DocumentElement

                    

                    # Add the Translation SourceType namespace to the namespace manager.

                    $nsmgr.AddNamespace("tra", "http://schemas.microsoft.com/LiveSearch/2008/04/XML/translation")

            

                    [System.Xml.XmlNodeList] $results = $root.SelectNodes("./tra:Translation/tra:Results/tra:TranslationResult", $nsmgr)

            

                    $row.item($index) = $results.item(0).InnerText

                }

            }

        }

    }

    

    $excelData

}

 

 

 

# Steps to make this work for your application"

#

# 1: Review the "TextResourceTranslation.xlsx" spreadsheet in this project to 

#     get a good feeling for how this script works with the column/row values

#     within it.

# 2: Create your own translation spreadsheet or modify the one provided

# 3: Substitute location path values below to match your project location

# 4: Substitute appropriate filenames

# 5: Substitute the name of the spreadsheet ($sheetName - this is the name from

#     the Excel workbook. In other words, the name on the tab at the bottom of

#     Excel.  In a new spreadsheet this defaults to "Sheet1".

# 6: Modify the $resxRegions array values.  The first value is the name 

#     of the resource (resx) file.  The second value is the column name of the 

#     appropriate language text within the spreadsheet.

 

 

 

# Root location of the spreadsheet and resx resource files

$projectPath = "D:\_App Dev\TechReady9_DEV320\TechReady9_DEV320"

# Filename of the Excel Translation spreadsheet

$excelDataFile = "TextResourceTranslation.xlsx"

# The name of the Excel sheet with the translation data in it

$sheetName = "TextTranslation`$"    

 

# Array of arrays: (region, Excel column name)

# For this script, the region is also used as the resource file (resx) filename

# Note: if the region is left blank ("") then it will become the default resource file

$resxRegions = @(("", "enUS"), ("es-ES", "esES"), ("ja-JP", "jaJP"))

 

# These are the languages for each of the DataSet Translations - 

# This is for use within the Bing auto translation feature

# If there are cells in the Excel spreadsheet that are empty, they will be filled

#  in with a call to Bing's translation service.

$translationLanguages = @("En", "Es", "Ja")

$appId = "Insert your Bing Application ID Here"

 

# This is the subfolder where the resx files are to be found/stored

$resxSubfolder = "Properties"

 

# This creates the full path for the Excel translation data file

[string] $excelFullPath = [System.IO.Path]::Combine($projectPath, $excelDataFile)

 

# Get the data from Excel and store in a dataset

[Data.DataSet] $excelData = Get-ExcelLanguageData $excelFullPath

 

# Perform translations using Bing Translator.  Only empty cells

#  in the non-default language columns will be translated

#  Note: This DOES NOT modify the Excel Spreadsheet.  It does the

#  translation and updates the resource files ONLY.  This step

#  is NOT required to generate resource files.  It is here to

#  "fill in the blanks" when the source spreadsheet does not 

#  have all cells filled in.  If you do not wish to use it,

#  comment the next line.

$excelData = Translate-Table $excelData $translationLanguages

 

# Loop through the regions and build the resx resource files

#  extracting the data from the Excel spreadsheet 

#  and modifying/adding to the xml in the resource file 

foreach($regionData in $resxRegions) {

 

    [string] $resxFilename = ""

 

    if ($regionData[0] -eq "") {

        $resxFilename = "Resources.resx"

    } else {

        $resxFilename = "Resources.$($regionData[0]).resx"

    }

    

    # Create the full path to the resource file

    [string] $resxFullPath = [System.IO.Path]::Combine($projectPath, $resxSubfolder)

    $resxFullPath = [System.IO.Path]::Combine($resxFullPath, $resxFilename)

    

    # Use the dataset to generate the resource 

    Create-ResxFile $resxFullPath $excelData $regionData[1]

}

 

Get Adobe Flash playerPlugin by wpburn.com wordpress themes