XSS Attack your database to detect missing Output Encoding

.Net, Improve Your Code, Secure your code, Security, XSSAttack No Comments

Cross Site Scripting (XSS) Attacks must be one of the most popular type of attacks of websites these days with maybe only SQL Injection attacks getting more attention that that.

XSS Attacks

The concept is quite simple for Persistent Attacks. User A attacks a website and via some vulnerability, it manages to push an XSS script into the storage (some database most of the times) of the website.

User B looks at the attacked website and part of the content delivered by the website the injected XSS from the User A is delivered to User B. Now, depending on what the script was, User A could have a great deal of control on what User A does and sees. From session stealing to running operation on behalf of User B.

These type of attacks can be devastating for a website with one of my preferred examples being the Samy is my hero XSS attack on MySpace back in 2006

Almost every year I talked at Microsoft TechEd I had to say something about XSS and how dangerous it is and I’m always surprised that very few people know about it and try to take measures to avoid XSS attacks on their websites.

Mitigations

Some of the mitigations factors are Input Validation, Input Filtering and Output Encoding. None of this mitigations have any value if used alone or incompletely.

Input Validation and Input Filtering are mandatory but don’t have them as your only protection. There are tools out there that can try to automatically throw hundreds of attacks at your application to see how it responds to input validation.

However I think Output encoding is the most critical one as the other ones are just security gates that sooner or later someone will be able to bypass one way or another. Output encoding is also the hardest one to test as you have to verify that your website properly encodes every bit of data that is/was/will be potentially modifiable by a malicious user.

XSS Attack Tool

This is where the XSS Attack Tool comes into place. The tool will simulate an attack on your database and update up to 5000 rows in every table and replace your strings in your database with random XSS attacks. Just imagine a malicious user had direct access to your database and tried to to his best to XSS attack your site.

After you run the tool go and browse your website. Considering the large number of XSS scripts injected in your database you should expect some of the scripts to surface in the UI as potential attacks. This will allow you to find and encode all the places where data could surface. The tool is not intended to cover every scenario possible so using the tool does not guarantee that your website is not vulnerable, however it’s yet another tool in the arsenal we are required to have to better protect our sites.

Usage

  1. Download XSS Attack 1.0.
  2. Unzip
  3. BACKUP your database.
  4. Now BACKUP your database again. Are you sure you did that? Do it one more time just to be sure you can get back to it.
  5. Modify the “Database” connection string in the XSSAttack.exe.config to point to your database
  6. Enter any Schema, Table or Column name (format Schema, Schema.Table or Schema.Table.Column) in the ignore field in the config file to let the tool know those should be ignored.
    • Ignore tables like System Properties tables that contain properties required for your site to run or Reference Data tables that can’t be modified or never surface back to the user
  7. Run the tool and watch it update your database.
    • The tool might fail to update some columns every now and then or fail if you have some custom indexes or referential integrity.
  8. Go back to your website and browse. If some scripts make it to the surface un-encoded you should see it pretty quickly.
  9. Note that you might have to do some manual updates to your database to get your site up and running again (e.g. fix your user name so you can login or your password).
  10. If ALL your website still looks good run the tool few more times and try again.

Note: The tool DOES NOT INSERT any data in your database. It only updates existing rows so make sure your database has a good sample set of your data that you can use for testing.

Demo

My current demo site, a GUID Store (just in case anyone wants to buy a GUID):

Running the tool:

Loading the website:

And few more alerts and errors.

I did a restore of the database and tried a new run of the tool and got different results:

The tool might give you false positives, e.g. modify columns that can never be modified by anyone and the results of which should not be encoded or not manage to create injections so take it’s results with a bit of salt. It’s for you to decide what a good result is and maybe try restore your database to a known state, configure the tool to ignore more columns and re-run the process.

Note: I’m not responsible for any loos, damage, security issues, fame or whatever before, during or after using this tool. This is a tool for developers and security reviewers so treat it as such. Make sure you backup your database before running this tool and make sure you never run this on a production system. The tool injects scripts from other websites that could be malicious to your browser or OS.

If you do find the tool useful then drop me an email at corneliu at acorns dot com dot au to let me know how you used it.

Download XSS Attack 1.0.

Merging WPFThemes with your own styles

.Net, WPF No Comments

Last few months I’ve been working on a WPF and as my design skills are close to zero I’ve decided to the the app a bit of sexiness by using the WPF goodness of Themes. Best place to start for that are the WPFThemes on CodePlex.

Sample from one of the themes:

  1. <!– Button –>
  2. <Style x:Key="{x:Type Button}" TargetType="{x:Type Button}">
  3.     <Setter Property="IsEnabled" Value="true"/>
  4.     <Setter Property="IsTabStop" Value="true"/>

Applying the standard themes to your project is pretty straight forward if you want to use 100% of the theme:

  1. public partial class ApplicationInstance : Application
  2.     {
  3.         protected override void OnStartup(StartupEventArgs e)
  4.         {
  5.             ThemeManager.ApplyTheme(this, "ShinyDarkTeal");
  6.             [...]
  7.         }

This works a treat however because the default styles are all keyed on the types there is no way for you to “customize” the styles in the themes and add new properties to them without modifying the original themes (which would create a maintenance nightmare and no chance of you ever merging with new version of WPFThemes) or use named styles which would imply you have to define the style of each element in all your code.

Option 1: Modify original theme:

  1. <Style x:Key="{x:Type Button}" TargetType="{x:Type Button}">
  2.     <Setter Property="IsEnabled" Value="true"/>
  3.     <Setter Property="Height" Value="50" /> <!– my custom property –>
  4.     <Setter Property="IsTabStop" Value="true"/>

Proper maintenance nightmare.

Option 2: Create named style:

  1. <Style x:Key="MyButton" TargetType="{x:Type Button}" BasedOn="{StaticResource {x:Type Button}}">
  2.     <Setter Property="Height" Value="50" />
  3.     <Setter Property="MinWidth" Value="90" />
  4. </Style>

Ugly as you need to use named keys through your application.

Option 3: Override it as window level not application level

  1. <Window.Resources>
  2.         <Style TargetType="{x:Type Button}" BasedOn="{StaticResource {x:Type Button}}">
  3.             <Setter Property="Height" Value="50" />
  4.         </Style>

Ugly as you have to maintain it for each window.

Note: You can’t use option 3 at application level as the BasedOn will be applied based on the standard style not the themed one.

None of the two options sounded like a good option so when all I wanted to do is be able to set default Width or Height of my buttons:

  1. <Style x:Key="{x:Type Button}" TargetType="{x:Type Button}">
  2.     <Setter Property="Height" Value="50" />
  3.     <Setter Property="MinWidth" Value="90" />
  4. </Style>

Wouldn’t it be nice if we could no “use the theme as it’ given to us” but “merge the theme” into our code as have it as the base of what we want to build on top of it.

Merging themes and overwriting via target inheritance

The only other option that I found was to do a custom load of the theme file as named keys and not as types even if they are defined as types and force my custom styles to inherit from the theme styles.

First the code:

CustomStyles.xaml – my custom (master) styles

  1. <Style x:Key="{x:Type Button}" TargetType="{x:Type Button}">
  2.     <Setter Property="Height" Value="50" />
  3.     <Setter Property="MinWidth" Value="90" />
  4. </Style>

x:Key is x:Type Button so I want this style to get applied to all my buttons.

Theme.xaml – default theme from WPFThemes

  1. <Style x:Key="{x:Type Button}" TargetType="{x:Type Button}">
  2.     <Setter Property="IsEnabled" Value="true"/>
  3.     <Setter Property="IsTabStop" Value="true"/>

These are the styles from the themes. We don’t want to modify them but we would like to merge them with our Custom Styles.

Merge code:

  1. private void MergeThemeFile()
  2. {
  3.     ResourceDictionary themeDictionary = ThemeManager.GetThemeResourceDictionary("BlueGlossyControls");
  4.     
  5.     string customStylesXaml = @"/myassembly;component/Resources/Styles/CustomStyles.xaml";
  6.     ResourceDictionary customStyles = Application.LoadComponent(new Uri(customStylesXaml, UriKind.Relative)) as ResourceDictionary;
  7.     
  8.     // Force remove of the custom styles dictionary so it does not get loaded
  9.     DropDictionary(customStylesXaml);
  10.  
  11.     ResourceDictionary finalDictionary = new ResourceDictionary();
  12.  
  13.     // we have two dictionaries, try to make the customStyles be the master by inheriting them from the theme
  14.     // update this, drop the xType if we already have some types and merge
  15.     foreach(DictionaryEntry resourceEntry in themeDictionary)
  16.     {
  17.         if (resourceEntry.Value is Style && resourceEntry.Key is Type)
  18.         {
  19.             Type themeKeyType = resourceEntry.Key as Type;
  20.             Style themeStyle = resourceEntry.Value as Style;
  21.             
  22.             // resource for a specific type, try to merge with customStyles
  23.             var localStyle = customStyles[resourceEntry.Key] as Style;
  24.             if (localStyle != null)
  25.             {
  26.                 // make local style inherit theme style
  27.                 string themeKey = "Theme-" + themeKeyType.FullName;
  28.                 finalDictionary[themeKey] = themeStyle;    // save the theme as "Theme-[FullName]"
  29.                 localStyle.BasedOn = themeStyle;    // make local style based on the Theme one
  30.                 finalDictionary[themeKeyType] = localStyle;
  31.                 continue;
  32.             }
  33.         }
  34.         finalDictionary.Add(resourceEntry.Key, resourceEntry.Value);
  35.     }
  36.     foreach (DictionaryEntry resourceEntry in customStyles)
  37.     {
  38.         if ( !finalDictionary.Contains(resourceEntry.Key))
  39.         {
  40.             finalDictionary[resourceEntry.Key] = resourceEntry.Value;
  41.         }
  42.     }
  43.     this.Resources.MergedDictionaries.Add(finalDictionary);
  44. }
  45. private void DropDictionary(string name)
  46. {
  47.     foreach (var resource in this.Resources.MergedDictionaries)
  48.     {
  49.         if (resource.Source.ToString() == name)
  50.         {
  51.             this.Resources.MergedDictionaries.Remove(resource);
  52.             break;
  53.         }
  54.     }
  55. }

This code will try to load the theme file and see if we have any keys as types that we override in the CustomStyles.xaml.

If we have a “conflict” we change the x:Key of the theme style to a named key and we change the BasedOn of the custom style to be based on the theme style. If there is no conflict we load the old theme.

This merges the two files like this:

  1. <!– Original Style From Theme: x:Key="{x:Type Button}" –>
  2. <Style x:Key="ThemeGlassyControls" TargetType="{x:Type Button}">
  3.     <Setter Property="IsEnabled" Value="true"/>
  4.     <Setter Property="IsTabStop" Value="true"/>
  5. </Style>
  6. <!– Original Style from CustomStyles based on the new Theme style–>
  7. <Style x:Key="{x:Type Button}" TargetType="{x:Type Button}" BasedOn="ThemeGlassyControls">
  8.     <Setter Property="Height" Value="50" />
  9.     <Setter Property="MinWidth" Value="90" />
  10. </Style>

We then load the new combined style.

This get the best of the two words out there combined. We can apply a theme while also overwriting/merging parts of it.

Merged styles: WPFThemes Gradients, My Width/Height

Now the next challenge is to be able to override part of the content. I’d like my buttons to have two lines of text. One with the text and one with the shortcut defined via a dependency property. I’d like to keep the theme contents of the button and be able to add to that in a very simply way.

Mazda6.Net CarPC: Putting it all together

CarPC, Personal No Comments

Intro

As a follow up from a previous post on Mazda6.Net CarPC: Getting the parts I’ve managed to purchase most of the parts including the Double-Din case from www.bybyte.com in which I had to fit the screen (and in my plan the mother board and HDD as well).

The Via M10000 board fits perfectly in the double-din case with just 3cm sticking out at the back which kind of worried me but I hoped I would have enough clearance in the car for those extra 3cm.

Fitting the motherboard almost worked according to the plan except that the power plug for the Via board is to on the side of the board thus there is not enough clearance next to it to plug the M3 ATX power adapter. Thus I’ve got stuck and dropped the project for several months until I found out from one of my customers that I can buy a 20-pin ATX Extension Cable that will get me unstuck by allowing me to move the M3 ATX power anywhere I see if.

With a new $10 ATX Extension cable in my hands I’ve decided it’s the right time to re-start (or just finish) my CarPC.

Couple of night of work to put all the bits an pieces together and voiala, CarPC working on the bench running WinXP. WinXP? Gosh, I can’t run my car on WinXP, so I started to install Windows 7 on the 5 years old Via M10000 using a USB CD Drive via the slow USB 1.1 ports. Installation time: 2.6h.

Upgrade: From Via to Intel

Fallback to Plan A.2. Buy a new sleek Intel Atom Mini-ATX board: Intel D945GCLF2 Motherboard – Dual-Core Intel Atom 330(1.6GHz), 945GC, 2xDDR2-533, 1xPCI, 2xSATA-II, GigLAN, 5.1Chl, VGA/S-Video, Mini-ITX</i> @ $115.70 + 1 x Corsair 2048MB PC2-5300 667MHz DDR2 RAM – 5-5-5-15 @ $56.10 ea.

Install:

  • Windows 7: Installation time: 18 minutes.
  • Drivers
  • Centrafuse
  • BlueSoleil Bluetooth
  • GPS Drivers (I can’t get the GPS working)
  • Virgin Mobile 3G Drivers

Few days later my CarPC is installed and working on the test bench, time to get a booking to the Car Audio Installation Specialists.

Test bench installing Windows 7

IMG_0256

Few more days of work to get it installed in the double-din case behind the touch screen:

View from above

DSC01851

The (dodgy) carton protection is for the M3 ATX power source.

View from the side

DSC01853

Big day comes that the system is almost all running and decide to use my last day of paternity leave to go and get the beast installed. Arranged an appointment with Druery Car Stereo to get it installed.

Last preparations

Evening before the appointment with the car stereo guys who were about to tear my car apart I did the last “in-car” tests.

DSC01855 DSC01864 DSC01869

The computer was powered from one cigarette lighter, the screen from another one.

Big day: Car Installation

Step 1: Tearing the car apart

DSC01870

This went very well but with unexpected results. The computer would not fit in the dashboard. It’s too long!

As you can see in this picture the motherboard is few centimetres longer than its case plus you have to add the length of the VGA plug and cable on top of this. That made the complete assembly about 6-8cm longer than a standard double-din, and that was just 3cm too long so the complete computer didn’t fit in the car.

DSC01852

I only had two options left:

  1. Put the car back together as it was and go home with no CarPC
  2. Install the screen and put cables so I can move the computer under the passenger seat.

Option 2 was the preferred one and we started to cut the original car wiring and get cables from behind the screen to under the passenger seat.

Step 2: Getting cables from behind the screen to under the passenger seat

DSC01871 DSC01872 DSC01873 DSC01874

Step 3: Install the screen in a custom Mazda fascia to make it look stock-standard

This also didn’t really work according to the plan as the screen was slightly smaller and the fascia was slightly bigger though they were both ‘universal standard’.

DSC01876

After a bit of fiddling we managed to put them together to make them look like one unit. It’s ok. It’s not perfect as there is a 3-4mm gap but it still looks good.

Day was over and had to take the car back home. Total billable time: 3.5h. Total time spent including searching documentation for cable and lunch: 6h.

We did no testing whatsoever of all the system or even if the cables are good so there was a high degree of chance of failure.

First tests

Yup. It failed. First test when I got home with a shiny screen in the car … failed. Screen was not powering on. It was completely blank which was not a good sign. Spent a good 3 nights with my head under the dashboard trying to track the power cable from the screen only to figure out that the power plug towards the car was not plugged in well. However that plug was quite deep in the middle of the dashboard, behind the panels I didn’t know/want to take out.

DSC01877

Here the screen is powered from an external source just to test that the screen works if it gets power.

DSC01879 DSC01878

I’ve started working to get the computer installed under the car seat and hit another set of challenges. The VGA cable from the screen is too short. It’s only sticking out about 3cm from the hole in the carpet, not enough to get it stable in the back of the PC. Now I’ll have to add a VGA extension.

More news soon.

Step by Step Database Encryption

.Net, Secure your code, Security No Comments

I’m sure there are a billion articles out there talking about using database encryption but I didn’t really find one that simply has a quick and simple guide.

Step 0: What do we need

The simplest approach is to use: a Master key on the database, a certificate and a symmetric key.

Step 1: Create Master Key

MSDN: The database master key is a symmetric key used to protect the private keys of certificates and asymmetric keys that are present in the database.

CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'Some Really long and complex password'

If you have no idea for a long and complex password you can use a free online password generator. Use a length of 28 or longer.

Step 2: Generate a Certificate

MSDN: […] when SQL Server generates a self-signed certificate, the private key is always created. By default, the private key is encrypted using the database master key. If the database master key does not exist and no password is specified, the statement will fail.

CREATE CERTIFICATE MyDatabaseCertificate
   WITH SUBJECT = 'Certificate For My Database',
   EXPIRY_DATE = '10/31/2011';

Step 3: Create the symmetrical key used for the encryption

MSDN: When a symmetric key is created, the symmetric key must be encrypted by using at least one of the following: certificate, password, symmetric key, asymmetric key, or PROVIDER.

CREATE SYMMETRIC KEY MyDatabase_Key WITH ALGORITHM = AES_256
    ENCRYPTION BY CERTIFICATE MyDatabaseCertificate;

Step 4: Give rights to the user(s) that will encrypt and decrypt using the certificate and key

In order to use (to open) the certificate for encryption and decryption you need to give view definition and control rights on the certificate to the user(s) that will do the operations.

GRANT VIEW DEFINITION ON CERTIFICATE :: MyDatabaseCertificate TO "MyDbUser"
GRANT CONTROL ON CERTIFICATE :: MyDatabaseCertificate TO "MyDbUser"

NOTE: the usage of quotation marks instead of normal apostrophes around the user name.

If you use Integrated Security your user would look like “DOMAIN\UserName”:

GRANT VIEW DEFINITION ON CERTIFICATE :: MyDatabaseCertificate TO "SERVER\MyWebUser"
GRANT CONTROL ON CERTIFICATE :: MyDatabaseCertificate TO "SERVER\MyWebUser"

Step 5: Encrypting and Decrypting Data

To encrypt and decrypt data you need to be able to open the symmetric key. In order to open a symmetric key you need to be in a stored procedure. You can’t open the certificate from a function or from code via TSQL calls.

Procedure to encrypt credit card numbers:

CREATE PROCEDURE [dbo].[usp_EncryptCreditCardNumber]
(
    @CreditCardNumber AS VARCHAR(16)
    , @Result VARBINARY(32) OUTPUT
)
AS
BEGIN
    SET NOCOUNT ON

    OPEN SYMMETRIC KEY MyDatabase_Key
    DECRYPTION BY CERTIFICATE MyDatabaseCertificate

    SET @Result = EncryptByKey(Key_GUID('MyDatabase_Key'), @CreditCardNumber)

    CLOSE SYMMETRIC KEY MyDatabase_Key
END

Procedure to decrypt credit card numbers:

CREATE PROCEDURE [dbo].[usp_DecryptCreditCardNumber]
(
    @EncryptedCreditCardNumber VARBINARY(32)
    , @CreditCardNumber AS VARCHAR(16) OUTPUT
)
AS
BEGIN
    SET NOCOUNT ON

    OPEN SYMMETRIC KEY MyDatabaseKey
    DECRYPTION BY CERTIFICATE MyDatabaseCertificate

    SET @CreditCardNumber = CONVERT(VARCHAR(16), DecryptByKey(@EncryptedCreditCardNumber))

    CLOSE SYMMETRIC KEY MyDatabaseKey
END

Step 6: Creating the table that will contain the encrypted data

Note: The encrypted column(s) have to be of type VarBinary of a length enough to keep the encrypted value of the data. Length depends on the algorithm used which in our scenario is AES_256. The AES works on blocks of 16 bytes so you should align your data to the next 16 size than the one that you have to encrypt.

So, lets just say you want to encrypt credit card details: You could create a table like:

CREATE TABLE [dbo].[CreditCardDetails](
    [CreditCardDetailsId] [int] IDENTITY(1,1) NOT NULL,
    [CreditCardNumber] [varbinary](32) NULL, -- Encrypted Credit Card
    [CreditCardLast4Digits] [char](4) NULL,
    [CreditCardExpMonth] [tinyint] NULL,
    [CreditCardExpYear] [int] NULL,
    [CreditCardName] [varchar](255) NULL,
    [CreditCardTypeId] [int] NULL,
 CONSTRAINT [PK_CreditCardDetails] PRIMARY KEY CLUSTERED
([CreditCardDetailsId] ASC)
)

You really want to keep the Last4Digits of the card in a separate column so you can present them in different UI items without having to decrypt the real credit card number.

Step 7: Inserting credit card details

Once we have the encrypt and decrypt procedure, the insert and retrieve become simple two step processes.

Save and encrypt:

CREATE PROC [dbo].[usp_CreditCardDetailsInsert]
    @CreditCardExpMonth tinyint,
    @CreditCardExpYear int,
    @CreditCardLast4Digits nchar(4),
    @CreditCardName varchar(255),
    @CreditCardNumber VARCHAR(16),
    @CreditCardTypeId int
AS
    SET NOCOUNT ON
    SET XACT_ABORT ON

    BEGIN TRAN

    DECLARE @EncryptedCreditCardNumber VARBINARY(32)
    EXECUTE dbo.usp_EncryptCreditCardNumber @CreditCardNumber, @EncryptedCreditCardNumber OUTPUT

    INSERT INTO [dbo].[CreditCardDetails] ([CreditCardExpMonth], [CreditCardExpYear], [CreditCardLast4Digits], [CreditCardName], [CreditCardNumber], [CreditCardTypeId])
    SELECT @CreditCardExpMonth, @CreditCardExpYear, @CreditCardLast4Digits, @CreditCardName, @EncryptedCreditCardNumber, @CreditCardTypeId

    COMMIT
GO

Retrieve and decrypt:

CREATE PROC [dbo].[usp_CreditCardDetailsSelectForPayment]
    @CreditCardDetailsId INT
AS
    SET NOCOUNT ON
    SET XACT_ABORT ON

    -- retrieve the encrypted credit card
    DECLARE @EncryptedCreditCardNumber VARBINARY(32)
    SELECT @EncryptedCreditCardNumber = CreditCardNumber
    FROM dbo.CreditCardDetails
    WHERE  [CreditCardDetailsId] = @CreditCardDetailsId

    DECLARE @CreditCardNumber varchar(16)
    -- decrypt
    EXECUTE [dbo].[usp_DecryptCreditCardNumber] @EncryptedCreditCardNumber, @CreditCardNumber OUTPUT

    -- select the other bits
    SELECT [CreditCardDetailsId], [CreditCardExpMonth], [CreditCardExpYear], [CreditCardLast4Digits], [CreditCardName], @CreditCardNumber AS CreditCardNumber, [CreditCardTypeId]
    FROM   [dbo].[CreditCardDetails]
    WHERE  [CreditCardDetailsId] = @CreditCardDetailsId

GO

PLEASE  make sure that you have another procedure for retrieving the non-critical data like name and last 4 digits of the card for display purposes without having to decrypt the credit card.

The only reason why you would need to decrypt the credit card is to send it to a payment gateway to process a transaction.

Do you want to “change” your Optus details? Just login. Here are the usernames & passwords.

Security 3 Comments

For reasons I’ll detail in a separate post I had to enter today in an Optus shop.

And, in a pure display of ignorance to any basic security measures, they had all the usernames and passwords for all the systems where the dealers could connect and do changes to accounts, create new accounts or remove them printed and stuck at the top of the screen.

An no, this was not on a computer somewhere in the back of the store but on a computer in the middle of the room full of people.

optus 

(I’ve reduced the size and quality of the image enough to have all the information impossible to read.)

Not only that I took this picture with my mobile but I managed to get friendly enough to play with the computer after the dealer logged in to a system that I could use to change my personal details (or anyone else’s details). I even did a view source on the (poorly coded) code and tried to hack some URLs together to avoid a post back on the page.

Uh, who, me? Never…

So where do you keep your passwords?

Microsoft SDC Open Day

Personal, Readify No Comments

On Monday the 6th of April Microsoft Australia organized the SDC Open Day 2009 event (SDC stands for [Microsoft] Software Development Centre, part of the Microsoft Services (Australia).

For over six years, Microsoft Australia’s Solutions Development Centre has been successfully delivering complex custom-developed applications for customers. We follow a set of processes that have enabled us to build high-performing teams delivering quality solutions. At the SDC Open Day we talked about how we have achieved this success, sharing our processes and techniques with an audience of software development professionals.

In the presentations below we provide an overview of the SDC, a day in the life for the team and details on how we approach each of the key disciplines in an SDC project team.

Together with few others like Tom Hollander, Prasadi de Silva (they are Microsoft so they don’t really count), , Bruce, Sarah from DevTest and Simon and Emma from Avanade we were invited to do talks on different aspects of the software development process in the SDC from Project Management, Development, Testing to Setup and Deployment.

The videos from the day are all published now. The fifth one in the list is my presentation :) however I can’t seem to be able to play it as I continuously get an error.

Leaving Readify: Got a new job & new responsibilities

Personal, Readify 3 Comments

It’s public now that as of the 15th of May I’ve resigned from the position of Senior Consultant in Readify.

I’ve worked with some of the best minds in the industry from Darren, Mitch, PaulG and PaulS (who is still Temporarily Offline), Philip, Damian, Tatham, Francois (who’s now enjoying his trip around the world), Aymeric and lots, lots more.

I’ve worked with some great people from outside Readify like Tom Hollander, Rocky Heckman, Bruce McLeod, to mention only a few..

I think this was the best, coolest and most enjoyable job I’ve ever had, with involvement in various projects from fixing simple (sometimes crappy) VB.Net apps to technical lead of large projects, performance reviews and improvements, security reviews and guidance, threat modelling new or existing systems, architecture and design of new systems, architecture reviews, complete or partial refactoring and rewrites of applications, mentoring, guidance, training, talking, presenting or attending loads of conferences, whinging, crying (not really) and contesting or challenging technical decisions (most of the times offering alternatives), building awesome relationships with my teams and my customers (oh, well, at least that’s what I believe I did :) )

Even more I also received an MVP award in Development Security.

I’ve loved my position and the people I’ve been working with.

But now it’s time to move on to a new challenge and a new role. I’ve accepted a new role as a (full-time) Solutions Architect and 2IC with Class Financial Systems. More or less I’m going back into financial systems where most of my background and experience was before I’ve joined Readify.

I’ll have a great new set of challenges from leading a new team to delivering a new project from zero to hero (or great success). I kind of started to forget what it means to be part of a project form end to end. I’ll also try to use this opportunity to learn, apply, present and (cross-fingers) be approved as a Microsoft Certified Architect: Solutions.

I truly hope I’ll keep in good contact with all Readify, Microsoft, friends and partners and I hope to see most of you at user groups, TechEd, CodeCamp, MVP Summit and other conferences.

WCF Dynamic Client Proxy – Implementing IDisposable

WCF, WCF Dynamic Proxy 1 Comment

As my good friend and Readify colleague Buddhike observed the generated WCF Dynamic Proxy was not implementing IDisposable. This is a bit scary as I was quite sure I had that implementation prepared done already so all I can image is that for some reason (no source control?) I’ve been working on an older code base.

So, here is a new implementation that is implementation that implements IDisposable. The only trick is that you have to cast the received proxy to IDisposable in order to use it in a using statement. As a difference from the WCF implementation, the dynamic proxy does not throw at all from the implementation of the IDisposable so it’s safe to use in a using statement:

IService1 service = WCFClientProxy<IService1>.GetReusableFaultUnwrappingInstance("Service1");
using (service as IDisposable)
{
    service.MyOperation1("a", 0);
}

Please download the new version of WCF Client Proxy 1.3.1.

WCF Dynamic (ClientBase) Proxy part three: Connection Pooling and Automatic Disposing

.Net, WCF, WCF Dynamic Proxy 1 Comment

A while ago I published a small and neat Dynamic Proxy that could be used to automatically create for you implementation of the (WCF) ClientBase<T> so you would not have to generate that from the service interface and hand-coded or even bother to maintain.

One of it’s great advantages was that you could ask for a “Reusable” proxy which was basically a wrapper around the proxy that you didn’t have to close/dispose in case of a fault.

As you might know WCF requires that you dispose of the proxy if there is any type of fault detected. Doing this is a pain in any type of code as you have to manage that connection life-time while actually all you care most of the times is that you talk to the service and not the life-time of your connection. The WCF Client Proxy was also doing this management for you allowing you so simply focus on your business and not on opening/closing your connections.

However there are moments in which you also want to specifically close your proxy for example using a using statement. Because the WCF Client Proxy was returning you the exact interface that you requested and if that interface was not implementing IDisposable you had to reside on a trick and cast the received proxy to IDisposable (as the proxy generates that behinds the scenes for you anyway) and use it like this:

IService1 service = WCFClientProxy<IService1>.GetReusableInstance("Service1");
using((IDisposable)service)
{
    service.MyOperation1();
}

This was again a bit ugly as you have to care about the connection and you can’t just use it as any other interface. You do have to be aware that your interface represents a WCF service that you want to dispose of.

One other potential scenario that several people hit is in websites that talk to external WCF services and you use the same service from within the same page several times (for example from different controls). In practice in this scenario you will open/close the same proxy multiple times during the lifetime of the page.

Connection Pooling

In order to simplify the management of WCF connections during the lifetime of a webpage or even a WCF Operation Call and reduce the overhead of caring to dispose of proxies I’ve now added a ProxyConnectionPool class to the dynamic proxy that can be enabled to automatically pickup all the created WCF connections and pool them (as in return the same one back to you if you use it from the same thread and it’s safe to reuse).

To enable the connection pooling all you have to do is:

ProxyConnectionPool.EnableConnectionPool = true;

Now every time you do a call do GetReusableInstance or other methods on the WCFClientProxy the connection you receive might be a pooled one (same tread only pooling) or a new one just registered with the pool. This should improve your performance as creating WCF Proxies is an expensive process that you want to avoid as much as possible.

Disposing the pool

Once you finish your work on that thread you can simply ask the connection pool to dispose all the WCF connections from the pool using:

ProxyConnectionPool.Current.Dispose();

Automatic disposing

If you use WCF inside a website to call other WCF services you should then simply include the following module in your web.config so you get automatic disposing of all WCF connections created on each page request (with peace of mind included).

<system.web>
    <httpModules>
        <add name="WcfConnectionPool" type="ACorns.WCF.DynamicClientProxy.Pool.WcfConnectionPoolHandler,ACorns.WCF.DynamicClientProxy"/>
    </httpModules>
</system.web>

This module will automatically enable the connection pool for you and make sure all the WCF proxies created are nicely disposed at the end of each page request. This truly allows you to focus on writing your business code and not bothering about connection management of your WCF services. Your code will now become:

IService1 service = WCFClientProxy<IService1>.GetReusableInstance("Service1");
service.MyOperation1();

Let someone else deal with the fact that you use WCF :)

Licence

I was requested several times about the licence this code is. Here is my official statement:

This package is provided "AS IS," without express or implied warranty of any kind, and may be used and modified.

This package may be used in corporate applications without any pretence.

However I would appreciate if you would drop me a line to let me know you are using it to corneliu at acorns.com.au or even donate some money by PayPal to the same address :)

Download

Here is the latest version of the WCF Client Proxy 1.3.0

Writing a super fast deep-property reader/writer using delegates

.Net, Visual Studio 3 Comments

For a while now I’ve been writing to write a non-reflection based property reader that allows me to read properties of an object in a deep hierarchy.

So lets say you have the following simple class hierarchy:

public class FirstLevel
{
    public string Leaf { get; set; }
}
public class SecondLevel
{
    public FirstLevel First { get; set; }
}
public class Root
{
    public SecondLevel Prop { get; set; }
}

Then you want to read the Leaf from the FirstLevel by looking at the Root level and all you have is the path to the property: “SecondLevel.FirstLevel.Leaf”.

Using reflection is a simple process of recursively going through the object, finding the proper property doing a GetValue on it and then repeating the process until you find your property. This is ok but it’s very slow. Very very slow.

Your other two alternatives if you don’t want the bear the hit of reflection is to code emit a method that would resemble something like this:

public static string GetLeaf(Root root)
{
    SecondLevel secondLevel = root.Prop;
    if ( secondLevel != null )
    {
        FirstLevel firstLevel = secondLevel.First;
        if ( firstLevel != null )
        {
            return firstLevel.Leaf;
        }
    }
    return null;
}

Or use dynamically created delegates to map to the get_Property and set_Property methods generated for each property.

 

using System;
using System.Reflection;

namespace ACorns.Utils
{
    public interface IPropertyAccessor
    {
        object GetValue(object target);
        void SetValue(object target, object value);
    }

    public interface IPropertyAccessor<TargetObject, FinalProperty>
    {
        FinalProperty GetValue(TargetObject target);
        void SetValue(TargetObject target, FinalProperty value);
    }

    /// <summary>
    /// Super-dooper, super-fact deep property extractor.
    /// You can use it to get/set properties deep in an object hierarchy without using reflection.
    /// Please cache the returned IPropertyAccessor if you want to reuse it.
    /// Good performance is only achived with cached IPropertyAccessor(s)!
    ///
    /// Usage: IPropertyAccessor accessor = PropertyExtractor.GetAccessor(typeof(Root), "Prop.First.Leaf", true);
    /// accessor.GetValue(target);
    /// </summary>
    public static class PropertyExtractor
    {
        public static IPropertyAccessor<TargetObject, FinalProperty>
            GetAccessor<TargetObject, FinalProperty>(Type targetType, string propertyNames)
        {
            return GetAccessor<TargetObject, FinalProperty>(targetType, propertyNames, true);
        }
        public static IPropertyAccessor<TargetObject, FinalProperty>
            GetAccessor<TargetObject, FinalProperty>(Type targetType, string propertyNames, bool throwOnNull)
        {
            IPropertyAccessor internalPropertyAccessor = GetAccessor(targetType, propertyNames, throwOnNull);

            Type accessorType = typeof(TypedPropertyAccessor<,>).MakeGenericType(typeof(TargetObject), typeof(FinalProperty));
            IPropertyAccessor<TargetObject, FinalProperty> propertyAccessor =
                (IPropertyAccessor<TargetObject, FinalProperty>)Activator.CreateInstance(accessorType, internalPropertyAccessor);

            return propertyAccessor;
        }
        public static IPropertyAccessor GetAccessor(Type targetType, string propertyNames, bool throwOnNull)
        {
            string[] deepPropertyNames = propertyNames.Split('.');
            IPropertyAccessor internalPropertyAccessor = GetAccessor(targetType, deepPropertyNames, 0, throwOnNull);
            return internalPropertyAccessor;
        }

        private static IPropertyAccessor GetAccessor(Type targetType, string[] deepPropertyNames, int level, bool throwOnNull)
        {
            string property = deepPropertyNames[level];

            PropertyInfo propertyInfo = targetType.GetProperty(property);

            // Create a delegate to a get_ method. The delegate looks like
            // Func<TargetType, PropertyType> func to a property like class TargetType { public PropertyType { get; } }
            Type getterDelegateType = typeof(Func<,>).MakeGenericType(targetType, propertyInfo.PropertyType);
            Delegate getDelegate = Delegate.CreateDelegate(getterDelegateType, propertyInfo.GetGetMethod());

            IPropertyAccessor accessor;

            level++;
            if (level < deepPropertyNames.Length)
            {
                // Recursive detect the down the property
                IPropertyAccessor nextLevelAccessor = GetAccessor(propertyInfo.PropertyType, deepPropertyNames, level, throwOnNull);

                Type accessorType = typeof(PropertyAccessor<,>).MakeGenericType(targetType, propertyInfo.PropertyType);
                accessor = (IPropertyAccessor)Activator.CreateInstance(accessorType, getDelegate, nextLevelAccessor, throwOnNull);
            }
            else
            {
                Type setterDelegateType = typeof(Action<,>).MakeGenericType(targetType, propertyInfo.PropertyType);
                Delegate setDelegate = Delegate.CreateDelegate(setterDelegateType, propertyInfo.GetSetMethod());

                Type accessorType = typeof(LeafPropertyAccessor<,>).MakeGenericType(targetType, propertyInfo.PropertyType);
                accessor = (IPropertyAccessor)Activator.CreateInstance(accessorType, getDelegate, setDelegate);
            }

            return accessor;
        }

        #region TypedPropertyAccessor
        internal sealed class TypedPropertyAccessor<T, U> : IPropertyAccessor<T, U>
        {
            private readonly IPropertyAccessor _next;

            public TypedPropertyAccessor(IPropertyAccessor next)
            {
                _next = next;
            }

            public U GetValue(T target)
            {
                return (U) _next.GetValue(target);
            }

            public void SetValue(T target, U value)
            {
                _next.SetValue(target, value);
            }
        }
        #endregion

        #region Recursive Property Accessors
        internal sealed class PropertyAccessor<T, U> : IPropertyAccessor
        {
            private readonly Func<T,U> _readDelegate;
            private readonly IPropertyAccessor _next;
            private readonly bool _throwOnNull;

            public PropertyAccessor(Func<T, U> readDelegate, IPropertyAccessor next, bool throwOnNull)
            {
                _readDelegate = readDelegate;
                _throwOnNull = throwOnNull;
                _next = next;
            }
            public object GetValue(object target)
            {
                object result = _readDelegate((T)target);
                if (result == null)
                {
                    if (_throwOnNull)
                        throw new NullReferenceException("Property '" + _readDelegate.Method.Name + "' on '" +
                            typeof(T).Name + "' returned null.");
                    else
                        return default(U);
                }
                return _next.GetValue(result);
            }
            public void SetValue(object target, object value)
            {
                object result = _readDelegate((T)target);
                if (result == null)
                {
                    if (_throwOnNull)
                        throw new NullReferenceException("Property '" + _readDelegate.Method.Name + "' on '" +
                            typeof(T).Name + "' returned null.");
                    else
                        return;
                }
                _next.SetValue((U)result, value);
            }
        }
        internal sealed class LeafPropertyAccessor<T, U> : IPropertyAccessor
        {
            private readonly Func<T, U> _readDelegate;
            private readonly Action<T, U> _setDelegate;

            public LeafPropertyAccessor(Func<T, U> readDelegate, Action<T, U> setDelegate)
            {
                _readDelegate = readDelegate;
                _setDelegate = setDelegate;
            }
            public object GetValue(object target)
            {
                object result = _readDelegate((T)target);
                return result;
            }
            public void SetValue(object target, object value)
            {
                _setDelegate((T)target, (U)value);
            }
        }
        #endregion
    }
}

To use it you would request an IPropertyExtractor and then ask it to do a GetValue for you:

[TestMethod]

public void ExtractPropertyFromLeafFixture()

{

    Root r = new Root();

    r.Prop = new SecondLevel();

    r.Prop.First = new FirstLevel();

    r.Prop.First.Leaf = "original value";

    IPropertyAccessor accessor = PropertyExtractor.GetAccessor(typeof(Root), "Prop.First.Leaf", false);

    Assert.IsNotNull(accessor);

    object value = accessor.GetValue(r);

    Assert.AreEqual("original value", value.ToString());

    accessor.SetValue(r, "new value");

   
    value = accessor.GetValue(r);

    Assert.AreEqual("new value", value.ToString());

}

The beauty of this approach is that once the IPropertyAccessor is created (using Reflection) in the beginning,  the Get/Set on it is done via a set of direct delegate calls to the property and calls via the interface to the next level.

The callstack is thus very small and efficient:

image

Make sure you cache this accessor if you need it again at a later time as creating it is expensive.

The performance should be as good as you can get without using code emitting.

« Previous Entries