Thursday, June 30, 2016

Improving performance of your DB2 INSERT and UPDATE operations

On a project where we are using DB2, we have to do a lot of bulk inserts and updates. We investigated the bulkcopy option, but in the end we decided to switch to an alternative option supported by DB2; “chaining”.  Chaining will bundle a set of calls and send them in one package to DB2. This has a major speed improvement compared to inserting/updating a lot of rows.

Note: The main reason we switched to a different approach  is that the Bulkcopy option in DB2 has limited support for transactions.

Here is how to use it:

Remark: When using BeginChain/EndChain you cannot combine this with SELECT operations. You end up with exceptions similar to the following one:

{"ERROR [HY010] [IBM] CLI0125E  Function sequence error. SQLSTATE=HY010\r\nERROR [HY010] [IBM] CLI0125E  Function sequence error. SQLSTATE=HY010"}

Wednesday, June 29, 2016

Creating a release pipeline for your mobile applications

I’m currently working on a release pipeline for our mobile applications, making it easy to build, deploy and test the mobile solutions we are creating on top of the Xamarin platform.

Everyone who ever tried to publish a mobile application to a store(Apple AppStore, Google Play Store, Microsoft App Store,…) knows that it is cumbersome process requiring things like certificates, keystores, a lot of configuration work and so on… To optimize the process we went fully automated by using a combination of the following tools:

  • Visual Studio Team Services
  • Hosted Build Service
  • MacInCloud
  • Xamarin Test Cloud
  • HockeyApp

A great help during our journey was the DevOps blog series by Richard Woollcott:

Tuesday, June 28, 2016

VSTS - Build and Deployment service blade is empty in the Azure portal

As mentioned in my blog post yesterday, you can buy extra private build slots through the Azure Portal. However when I tried to do this, the Build and Deployment service in the Azure portal refuses to load and remains empty. I got the problem no matter if I try Chrome, IE, Firefox.

image

Anyone with the same problem?

Monday, June 27, 2016

VSTS Build–Private Build slots; you only get one for free!

By default when you are using VSTS(aka Visual Studio Online), you get the following extra services as part of your subscription:

Build and Deployment: Use this task-based service to create, queue, and monitor cross-platform builds. Use Hosted Agents that Microsoft runs, or Private Agents that you run so that you can install custom software.

  • Build(XAML)/ Build and Deployment: Create build definitions  and run them in Visual Studio Team Services. Configure builds to run on demand at specific intervals, or for continuous integration (CI). Builds are charged per minute for the actual amount of computing time used to build your project.

  • Cloud-based Load Testing: Create load tests using Visual Studio Ultimate 2013, Visual Studio Enterprise 2015, or later. Run those tests in Visual Studio Team Services. Load tests are measured and billed in virtual user minutes: the number of virtual users multiplied by the number of minutes that you set up for the load test run.

Your Visual Studio Team Services account includes free amounts of these additional services:

  • Build (XAML) / Build and Deployment: combined 240 minutes (4 hours) per month

  • Cloud-based Load Testing: 20,000 virtual user minutes per month

With the new Build and Deployment you get 2 things:

  • One hosted build agent that runs on a build server provided by Microsoft
  • One private build agent that you can install on your own hardware(virtual or physical)

I wasn’t aware of the second thing, so I got into troubles when I tried to add a second private build agent and ended up with some weird errors. Luckily it is easy to buy extra private agents through the Azure Portal. Here are the steps you need to follow: https://www.visualstudio.com/docs/setup-admin/team-services/get-more-build-or-load-testing-vs#AzurePortal

Click Paid. Select total number of agents. Save your changes

Remark: The hosted XAML build service will be retired by September 2016.

Friday, June 24, 2016

Nice quote–The inevitability of design

A quote by Douglas Martin in his book ‘Book Design: A practical Introducation’ but also applicable to software development and software architecture:

“Questions about whether design is necessary or affordable are quite beside the point: design is inevitable. The alternative to good design is bad design, not no design at all.”

Douglas Martin

Thursday, June 23, 2016

TFS Build–How to set log verbosity

After helping a customer upgrade to TFS 2015, they switched there existing build templates to the latest TFS (XAML) build process template(TfvcTemplate.12.xaml).

image

However after doing that, they noticed that they could no longer change the log verbosity of the build.

image

Starting from TFS (XAML) Build 2013 this is no longer necessary as the build diagnostics is always switched on. The easiest way to see the diagnostics is to browse to the web portal, open a build result and click on the Diagnostics tab:

BuildDiagnostics

Wednesday, June 22, 2016

TFS Build: Running your JavaScript unit tests during build

To run our JavaScript unit tests(you have them right?) inside Visual Studio we are using Chutzpah. Chutpzah is a JavaScript test runner which enables you to run unit tests using QUnit, Jasmine, Mocha, CoffeeScript and TypeScript(!).

Chutzpah has a test adapter for the Visual Studio Unit Test Explorer which makes it possible to run your JavaScript unit tests among your other tests.

This week I spent some time in making these tests also work on the build server. Here are some of the lessons I learned among the way:

Tip 1  - Add the Chutzpah nuget package as a reference to your unit test project

Add the Chutzpah nuget package to the project that contains your JavaScript unit tests. This NuGet package contains all required DLL’s and tooling to be able to run your JavaScript tests. By using this approach you don’t need to install the Chutzpah test adapter on your build server or specify the location of your test adapter. The TFS Build testing tools will automatically detect the required DLL’s and load the test adapter for you.

Tip 2 – Start using a chutzpah.json file

Chutpzah uses a lot of defaults out-of-the-box. But you have the option to tweak and configure almost everything. The Chutzpah test runner will search for this configuration file in all your test directories and multiple config files can be used.

Here is the config file that we are using:

More information: https://github.com/mmanela/chutzpah/wiki/Chutzpah.json-Settings-File 

Tip 3 – Remember the ‘IsTestFrameworkFile’ configuration option

As you can see in the config file above, we are using Angular at the moment. However the first time when we tried to run it using the chutzpah.json file, we got the following error:

“ReferenceError: ‘module’ is undefined”

This was caused by the fact the Angular and Angular.mock file were not yet loaded. You can fix this by setting the “IsTestFrameworkFile” option to “True”. This ensures that the references will be placed directly after the test framework files (like Jasmine.js) in the test harness.

Tuesday, June 21, 2016

JavaScript: const vs let vs var

With the introduction of EcmaScript 6 (EcmaScript 2015), 2 new keywords(among a lot of other language features) are added:

  • const: creates a read-only reference to a value. It does not mean the value it holds is immutable, just that the variable identifier cannot be reassigned.
  • let: declares a block scope local variable

Although these 2 keywords seems simple, there is still some debate in the JavaScript community when to use ‘const’ vs ‘let’ vs ‘var’. Originally ‘let’ was stated as ‘the new var’, but others are promoting ‘const’ as the default.

Here are some general rules that I follow:

  • If a variable won’t be reassigned I use ‘const’
  • If a variable may be reassigned I use ‘let’
  • If someone points a gun at my head and I have no other choice, I use ‘var’ Winking smile

Monday, June 20, 2016

Power BI free e-book

Microsoft released a free ebook about Power BI:

imageGet started quickly with Microsoft Power BI! Experts Alberto Ferrari and Marco Russo will help you bring your data to life, transforming your company’s data into rich visuals for you to collect and organize, allowing you to focus on what matters most to you. Stay in the know, spot trends as they happen, and push your business to new limits.

This free ebook introduces Microsoft Power BI basics through a practical, scenario-based guided tour of the tool, showing you how to build analytical solutions using Power BI. Read the ebook to get an overview of Power BI, or dig deeper and follow along on your PC using the book’s examples.

Download PDF: https://aka.ms/IntroPowBI/StndPDF

 

 

 

More information: https://blogs.msdn.microsoft.com/microsoft_press/2016/06/16/free-ebook-introducing-microsoft-power-bi/

Friday, June 17, 2016

SQL Server 2016 Temporal Tables

SQL Server 2016 introduces a new feature ‘Temporal Tables’. Temporal tables, also named system-versioned tables, allow SQL Server to automatically keep history of the data in the table. Before you typically had to maintain this yourself using triggers or other solutions to keep older versions of your data round.

Note that temporal tables are not a replacement for the change data capture (CDC) feature. CDC uses the transaction log to find the changes and typically those changes are kept for a short period of time (depending on your ETL timeframe). Temporal tables store the actual changes in the history table and they are intended to stay there for a much longer time.

When you want to create a new temporal table, a couple of prerequisites must be met:

  • A primary key must be defined
  • Two columns must be defined to record the start and end date with a data type of datetime2. If needed, these columns can be hidden using the HIDDEN flag. These columns are called the SYSTEM_TIME period columns.
  • INSTEAD OF triggers are not allowed. AFTER triggers are only allowed on the current table.
  • In-memory OLTP cannot be used

A table_history table will be automatically generated when you create a system-versioned table. You can also convert an existing history table to a temporal table.

Great feature! Smile

Temporal-HowWorks

More information here: https://msdn.microsoft.com/en-us/library/dn935015.aspx

Thursday, June 16, 2016

TypeScript error - Build: Unsupported locale 'nl-BE'.

As I’m building mostly enterprisy applications with large teams, tools like TypeScript are a lifesaver…until they start to break apart.

Last week after updating to a more recent TypeScript version, one of my colleagues got the following error message when trying to compile some TypeScript code:

vstsc    Error Build: Unsupported locale 'nl-BE'. MyApp.Web

A fix is already available in the nightly nuget package(see https://github.com/Microsoft/TypeScript-Handbook/blob/master/pages/Nightly%20Builds.md#using-nuget-with-msbuild for more details).

I preferred to use a workaround by copying C:\Program Files (x86)\Microsoft SDKs\TypeScript\1.8\en folder to C:\Program Files (x86)\Microsoft SDKs\TypeScript\1.8\nl-BE. TypeScript is using this folder to apply localization to the error messages, etc… Unfortunately if it couldn’t find a matching folder for your culture, you end up with a compiler error.

Wednesday, June 15, 2016

Soft Skills Engineering podcast

In my continuous quest to spend my (too long) commute time in a useful matter, I discovered the following great podcast that focus on the social aspects of IT:

Soft Skills Engineering podcast

softskillspodcast

A show about the "everything else" in software engineering: team work, career planning, interpersonal skills, and more.

Links:

Tuesday, June 14, 2016

SqlLocalDB command line tool

A few years ago, Microsoft created SQL Server LocalDB,  an execution mode of SQL Server Express targeted to developers. LocalDB installation copies a minimal set of files necessary to start the SQL Server Database Engine making it much easier to install and manage. Once LocalDB is installed, developers initiate a connection by using a special connection string:
Data Source=(LocalDB)\MyInstance;Initial Catalog=foo;Integrated Security=True;
When connecting, the necessary SQL Server infrastructure is automatically created and started, enabling the application to use the database without complex or time consuming configuration tasks. An instance of SQL Server Express LocalDB is managed by using the SqlLocalDB.exe utility.
LocalDB supports two kinds of instances: Automatic instances and named instances.
From MSDN:
  • Automatic instances of LocalDB are public. They are created and managed automatically for the user and can be used by any application. One automatic instance of LocalDB exists for every version of LocalDB installed on the user’s computer. Automatic instances of LocalDB provide seamless instance management. There is no need to create the instance; it just works. This allows for easy application installation and migration to a different computer. If the target machine has the specified version of LocalDB installed, the automatic instance of LocalDB for that version is available on the target machine as well. Automatic instances of LocalDB have a special pattern for the instance name that belongs to a reserved namespace. This prevents name conflicts with named instances of LocalDB. The name for the automatic instance is MSSQLLocalDB.
  • Named instances of LocalDB are private. They are owned by a single application that is responsible for creating and managing the instance. Named instances provide isolation from other instances and can improve performance by reducing resource contention with other database users. Named instances must be created explicitly by the user through the LocalDB management API or implicitly via the app.config file for a managed application (although managed application may also use the API, if desired). Each named instance of LocalDB has an associated LocalDB version that points to the respective set of LocalDB binaries. The instance name of a LocalDB is sysname data type and can have up to 128 characters. (This differs from regular named instances of SQL Server, which limits names to regular NetBIOS names of 16 ASCII chars.) The name of an instance of LocalDB can contain any Unicode characters that are legal within a filename. A named instance that uses an automatic instance name becomes an automatic instance.

To connect to an automatic instance, you can use the following connection string:
Server=(localdb)\MSSQLLocalDB;Integrated Security=true
To connect to a namend instance, you first need to use the SqlLocalDB.exe command line tool to create and start a named instance of LocalDB:
REM Create an instance of LocalDB  
SqlLocalDB create MyInstance
REM Start the instance of LocalDB  
SqlLocalDB start MyInstance
REM Gather information about the instance of LocalDB  
SqlLocalDB info MyInstance
Once created you can use the following connection string:
Data Source=(localdb)\MyInstance;Initial Catalog=foo;Integrated Security=True;
UPDATE: If you are using a SQL Server version below 2014, the connectionstring for the automatic instance is/was:

Server=(localdb)\v11.0;Integrated Security=true

Monday, June 13, 2016

Microsoft Midori: The operating system that would have changed everything

The last months, I spent some time reading up about Midori, a (new) OS created by Microsoft. Midori was incubated as an operating system written entirely in C# that achieved performance comparable with production operating systems, minus the security and reliability problems found in OSes written in C or C++.

Although Midori was abandoned, there are a lot of key lessons they learned along the way.

Joe Duffy, a development manager and language architect on Midori,  shared a series of a blog posts about his work on Midori:

  1. A Tale of Three Safeties
  2. Objects as Secure Capabilities
  3. Asynchronous Everything
  4. Safe Native Code
  5. The Error Model
  6. Performance Culture

More information about the history of Midori can be found here: http://www.zdnet.com/article/whatever-happened-to-microsofts-midori-operating-system-project/

Update: Joe Duffy did a session at QCon New York where he explained some of the lessons learned during the project.

Friday, June 10, 2016

JWT.io: Decode, verify and generate JSON Web Tokens

I’m currently spending a lot of time in implementing security using oAuth and OpenID. One of the things I had to start using were JSON Web Tokens(JWT).

JSON Web Token (JWT) is an open standard (RFC 7519) that defines a compact and self-contained way for securely transmitting information between parties as a JSON object. This information can be verified and trusted because it is digitally signed. JWTs can be signed using a secret (with the HMAC algorithm) or a public/private key pair using RSA.

JSON Web Tokens consist of three parts separated by dots (.), which are:

  • Header
  • Payload
  • Signature

A tool that helped me in debugging and testing these Web Tokens is JWT.IO. JWT.IO allows you to decode, verify and generate JWT.

image

You can take a token generated by your Security Token Service (like for example ADFS or IdentityServer) and paste it on the website. It will decode the token and show you the header and payload and allows you to verify the signature.

image

Thursday, June 9, 2016

Exception handling with Promises

There are some things you need to be aware of when handling exceptions in promises. If you are not careful an exception can be swallowed and the caller of the promise will never known an exception occured.

Therefore you should always take the following rules into account:

  • Add global exception handling to your promise chain by ending with a catch block.
  • Return a rejected promise if you want to maintain the exception in the promise chain.

So don’t do this:

Instead return a rejected promise in your catch block:

More information: http://odetocode.com/blogs/scott/archive/2015/10/01/javascript-promises-and-error-handling.aspx

Wednesday, June 8, 2016

NDC 2016–How to NOT do MicroServices

I’m currently at NDC Oslo following the Microservices with NServiceBus workshop by Jimmy Bogard. I got a lot of great insights and I’m impressed how NServiceBus helps you handle the complexity of a MicroServices architecture.

During the workshop Jimmy refered to the “Microservices: An application revolution powered by the cloud” article as an example on how to NOT do Microservices. Whoops! I don’t think that’s the way that Microsoft ment this article. Confused smile

Jimmy specifically refers to the way that the monolith application mentioned in the article is split out in multiple “micro”-services.

Breaking the monolith in microservices

As you can see in the image above, the microservices are oriented around technical concerns which is never a good idea when building a service architecture. Instead services should be oriented around business concerns and match your business processes.

He learned the hard way that the approach in the article didn’t work so don’t make the same mistakes.

Tuesday, June 7, 2016

Migrating from TFS Version Control to Git–Follow-up

A few months ago, I blogged about the Git-TF tool and Git-TFS tool I have been using to migrate TFSVC repositories. Last week I noticed a blog post by Richard Fennell from Black Marble where he describes in detail the steps needed to migrate a TFS VC team project to a Git team project. He not only talks about migrating the source code but also about migrating work items, retaining changesets, test plans and suites.

image

He is using the same toolset I have been using but provides a lot more detail. So certainly worth a read…

And of course, before I forget, here is the link to the article: https://www.microsoft.com/en-gb/developers/articles/week03aug14/migrating-a-tfs-tfvc-based-team-project-to-a-git-team-project/

Monday, June 6, 2016

Microsoft Azure–Redis Cache Advisor

With the latest Azure update, Microsoft gave some love to the Redis Cache implementation. One of the things that got added is the Redis Cache Advisor.

image

On your Azure Redis Cache instance, you’ll find a new Recommendations blade.

image

This blade is empty in normal situations but if any special condition occurs, like high memory usage, high server load, etc… an alert is displayed on the Redis Cache blade and the Recommendations blade will be filled with extra information on how to fix this(most of the time buy a bigger box Smile).

Recommendations

More information: https://azure.microsoft.com/en-us/documentation/articles/cache-configure/#redis-cache-advisor

Friday, June 3, 2016

Angular 2 changes

After upgrading my Angular 2 code to the latest version, my application started to fail with the following error message:

Template parse warnings:

"#" inside of expressions is deprecated. Use "let" instead! ("

            </thead>

            <tbody>

                <tr (click)="onSelect(application)" [ERROR ->]*ngFor="#application of applications">

                    <td>{{application.id}}</td>

            "): ApplicationsComponent@16:52

The ngFor syntax has changed. We now use let item of items instead of #item of items inside *ngFor.

For more information, check out the great post about ngFor by John Papa: https://johnpapa.net/angular-2-ngfor/

Thursday, June 2, 2016

Using a verbatim string to span multiple lines

Today during a code review, I noticed the following code snippet(I changed the code a little bit to proof my point):

Although I was happy to see the usage of a StringBuilder to optimize the string concatination process, in this case, as the full string message was static, it would have been easier to just use a verbatim string literal to create the multiline statement at once:

Wednesday, June 1, 2016

DDD Europe 2016–Videos

If you couldn’t be at the first DDD Europe conference, I have good news. The videos are online so you can see for yourself what you had to miss…

image

And even beter news, a second edition is announced for 2017 in Amsterdam: http://dddeurope.com/2017/