Thursday, April 17, 2008

Why do you prefix stored proc names with sp?

You don't call variable "x" in your C# code xVariable or var_x.

You don't call a table "Products" in your database ProductsTable or table_Products.

So why do you call your stored procedures sp_GetProducts?

Are prefixes really necessary in stored proc names? Isn't it obvious enough that it's a stored procedure?

Just wondering.

(If you have to prefix your stored procs, don't use "sp" because then they appear to be system stored procs...)

Monday, March 03, 2008

Summary of X.509 certificate file types

What is the difference between .spc and .cer? .pvk and .pfx?

As part of my ongoing mission to try to understand X509 certificates I wanted to figure out how to copy a test SSL certificate from one PC to another. The problem is makecert.exe generated a .cer file and I needed a .pfx file to import the certificate on the other server.

Use the -pe option with makecert.exe to create and install a certificate with an exportable private key. This should allow you to export the .pfx file from Certificates console.

Otherwise, use cert2spc.exe and pvkimprt.exe to convert the .cer and .pvk created by makecert.exe to a .pfx file:
makecert -pe -n "CN=My Root CA, O=Test, OU=For Testing Only!" -ss my -sr LocalMachine -a sha1 -sky signature -r -sv test.pvk test.cer
cert2spc test.cer test.spc
pvkimprt -pfx test.spc test.pvk
Or use pvk2pfx instead of the above:
pvk2pfx -pvk test.pvk -spc test.cer -pfx test.pfx -po pfx_password_here -f
X.509 File Types:
  • .CER = "DER" encoded binary X.509 Certificate file, e.g. as created by makecert. Contains only the public key.
  • .CRT = base-64 encoded X.509 certificate. (Not the same as a Microsoft standard base-64 .cer file...?)
  • .SPC = Software Publisher's Certificate. PKCS #7 standard that contains X.509 certificates. The cert2spc.exe utility converts a .cer file to .spc. .spc files are used as input to signcode.exe.
  • .PVK = private key file, e.g. as created by makecert.
  • .PFX = Personal Exchange file, contains certs with both the public and private key. PKCS#12 standard, binary DER encoded.
Some Windows certificate tools:
  • makecert.exe
  • pvkimprt.exe
  • cert2spc.exe
  • selfssl.exe
  • pvk2pfx.exe
  • certmgr.exe
See also:

Thursday, February 28, 2008

The use of IDisposable and finalizers

An explanation of how the Garbage Collector handles IDisposable and finalizers.

At my work we have lots of code like this in our ASP.NET apps:
class LogonPage : IDisposable
{
    public LogonPage()
    {
    }
    
    ~LogonPage()
    {
        this.Dispose();
    }
    
    public void Dispose()
    {
    }
    
    // rest of code here
}

In other words, a class that implements IDisposable with an empty Dispose method and a finalizer (aka destructor) that calls Dispose.

The intention of this code appears to be to somehow give a "hint" to the Garbage Collector to free the memory associated with this object when the "destructor" is called.

Implementing a finalizer in this case has no benefit at all, in fact, it probably has the opposite effect of what was expected.  It will not cause the GC to free the object's memory any earilier than an object not implementing a finalizer and it places additional load on the GC.

There are two problems with the assumption that implementing IDisposable and a finalizer will somehow affect the GC to free the memory earlier:

1. C# finalizers are not deterministic;

2. Objects with finalizers are kept in memory longer than objects without and require additional GC processing.

Non-determistic finalization

It is a common misconception that a C# destructor is similar to a C++ destructor.  In C# the "~" method is often called a destructor, but that's a misnomer.  It's not a destructor in the C++ sense.  Instead the "~" method is really just an override for Object.Finalize().  It was a mistake of the .NET implementors to even use the "~" character to indicate a finalizer.

In C++ the destructor is deterministic, in other words, the developer is responsible for allocating and freeing memory and destructor is called at an exact predictable time: e.g. when an object on the stack reaches the end of its lifetime (i.e. when it leaves the scope of the code block) or when an object on the heap is explicitly freed with the delete keyword.

But .NET finalization is "non-deterministic".  That means you don't know when and in what order the GC will call the finalizers and you have no control over when the object will be freed.  Unlike with C++ destructors, the programmer has no way of explicitly calling a .NET finalizer.

Generations and finalization

The .NET GC uses a generational algorithm.  This means the GC divides the managed heap in three logical sections, or generations, called gen 0, gen 1 and gen 2.  Each of these generations have a maximum size, initially 256kb, 2MB and 10MB respectively. New objects are usually allocated on gen 0.

When a new object is allocated and the gen 0 memory is too full to accomodate the new object, the GC will reclaim the memory allocated to unreachable objects by compacting the gen 0 memory.  Next the GC promotes all gen 0 objects to gen 1.  This empties gen 0 for new objects.

Similarly when the gen 1 heap becomes full, the GC reclaims unused memory and promotes the gen 1 objects to gen 2.

The GC performs collection more frequently on lower generations than higher generations.

The GC frees finalizable objects much later than those wihout finalizers.

When a finalizable object is created, a reference to it is added to the "finalization list".  When the GC determines that an object is no longer reachable, but it's in the finalization list, it's not freed immediately.  Instead the finalizable object is put on another special queue called the "f-reachable queue" and promoted to the next generation.  A special thread called the "finalizer thread" monitors the f-reachable queue and calls the finalizers when necessary.  These objects will eventually be freed the next time a gen 1 or 2 garbage collection is performed.  Therefore it takes at least two garbage collection cycles to free finalizable objects.

Recommended IDispoable pattern

Implementing IDisposable.Dispose has no effect on the GC at all.  It's little more than just a standard interface for providing a "Close" method on your class to allow the user to close/unload resources.

Use the following "Dispose pattern" when your class uses other managed classes that implement IDisposable (note a finalizer is not needed here).  This is the most common scenario.
class MyClass : IDisposable 
{
    private bool disposed = false;
    private FileStream file;
    
    public MyClass()
    {
        file = new FileStream(...);
    }
    
    public void Dispose()
    {
        if (!disposed)
        {
            file.Close();
            disposed = true;
        }
    }
    
    public void Close()
    {
        Dispose();
    }
}

If you have to implement IDisposable and a finalizer, use the recommended pattern as described in MSDN.
class MyClass : IDisposable 
{
    private bool disposed = false;
    private IntPtr file = IntPtr.Zero; // Unmanaged resource
    
    public MyClass()
    {
        file = ...; // allocate unmanaged resource
    }
    
    ~MyClass()
    {
        Dispose(false);
    }
    
    public void Dispose()
    {
        Dispose(true);
        GC.SuppressFinalize(this);
    }
    
    public void Close()
    {
        Dispose();
    }
    
    protected virtual void Dispose(bool disposing)
    {
        if (!disposed)
        {
            if (disposing)
            {
                // dispose managed resources
            }
            if (file != IntPtr.Zero)
            {
                // free unmanaged resource
                file = IntPtr.Zero;
            }
            disposed = true;
        }
    }
}

Summary

Only implement IDisposable if you have a reason to do so, e.g. if your class has member fields that implement IDisposable so you need to provide a Dispose method call Dispose on the member fields.

The simplest guideline regarding the use of finalizers is: don't use it at all unless your class directly wraps unmanaged resources (such as a native file handle) and you want to make sure that Dispose gets called at some point, which really almost never happens in a typical ASP.NET application.  Don't create empty finalizers.

A class implementing IDisposable doesn't always require a finalizer, but a class that implements a finalizer should always implement IDisposable.

This was just my interpretation of the GC process.  For a more complete and accurate description, see Jeffrey Richter's book "CLR via C#".

Also see:

Tuesday, February 05, 2008

Returning a byte array from C++ to Visual Basic 6

I had to figure out how to return a byte array from a C++ DLL to VB6 (using unmanaged C++ in Visual Studio 2005).

This is the general idea:

__declspec(dllexport) int __stdcall modifyArrayTest(SAFEARRAY** fileData)
{
 HRESULT hr;
 SAFEARRAYBOUND bound[1];
 if (*fileData != NULL)
 {
  TCHAR msg[128];
  _stprintf(msg, "Received %lu byte array, destroying it and reallocating", (*fileData)->rgsabound[0].cElements);
  OutputDebugString(msg);
  SafeArrayDestroy(*fileData);
 }
 else
 {
  OutputDebugString("Received null array, reallocating");
 }
 bound[0].cElements = 10; // 10 byte array
 bound[0].lLbound = 0;
 *fileData = SafeArrayCreate(VT_UI1, 1, bound);
 hr = SafeArrayLock(*fileData);
 if (hr)
 {
  OutputDebugString("SafeArrayLock failed");
  return -1;
 }
 unsigned char *data = (unsigned char*)(*fileData)->pvData;
 for (int n = 0; n < 10; n++)
 {
  data[n] = n;
 }
 SafeArrayUnlock(*fileData);
 return 0;
}


Now the function is imported as follows in VB6:

Private Declare Function modifyArrayTest _
Lib "ArrayTest.dll" (ByRef a() As Byte) As Long


And used like this:

Dim data() as Byte
Dim returnedBytes as Long
Call modifyArrayTest(data)
returnedBytes = UBound(data) + 1


Notes:
  • I found it useful to use OutputDebugString to log debug messages in my DLL code and then using DebugView from SysInternals to view the messages.
  • I noticed the DLL has to be compiled using ANSI (i.e. Character Set = Not Set) instead of Unicode (the default) to be able to pass strings back to VB6.
Reference: VB5DLL.TXT

Thursday, January 10, 2008

Testing SSL and Certificate Authentication using makecert

I wanted to set up a development environment, all on my local PC, that would allow me to do Certificate Authentication in an ASP.NET application.

Tools used:
  • IIS 5.1 / Windows XP
  • Web browser: IE 6.0
  • IIS Resources / WFetch for troubleshooting HTTP issues
The first step is to set up SSL on IIS. You need a server certificate for this, which you usually get from a Certificate Authority, but for development/testing purposes you just use makecert.exe (installed in the Visual Studio directory).

First we create a "Root CA" for our two certificates:
makecert -pe -n "CN=Dev Root Auth" -ss my -sr LocalMachine -a sha1 -sky signature -r "DevRootAuth.cer"
Open MMC. Add the Certificates snap-in for Local Computer.

Copy and paste the above "Dev Root Auth" certificate from the Personal path to the "Trusted Root Certification Authorities" path. Or use certmgr to copy it:
certmgr -add -all -c "DevRootAuth.cer" -s -r LocalMachine Root
Now create the server certificate (to use for SSL), using the first certificate created above as the Issuer:
makecert -pe -n "CN=localhost" -ss my -sr LocalMachine -a sha1 -sky exchange -eku 1.3.6.1.5.5.7.3.1 -in "Dev Root Auth" -is MY -ir LocalMachine -sp "Microsoft RSA SChannel Cryptographic Provider" -sy 12 localhost.cer
Note the "Extended Key Usage" value of 1.3.6.1.5.5.7.3.1 which indicates that this cert is for Server Authentication.

Now enable SSL: open Inetnet Information Services management console. Right-click Default Web Site and go to Properties -> Directory Security -> Server Certificate -> Assign an existing certificate. Select the "localhost" cert. Finish the wizard.

You should now be able to access pages on your localhost with HTTPS.

For example, create a directory C:\inetpub\wwwroot\SecureTest with a file named Default.aspx containing the following three lines:
Cert present: <%=Page.Request.ClientCertificate.IsPresent%>

Subject: <%=Page.Request.ClientCertificate.Subject%>
Open now open the file in IE as https://localhost/SecureTest/default.aspx. The output will show:
Cert present: False
Subject:
This verifies HTTPS is working. When you click the padlock icon in Internet Explorer you should see the "localhost" certificate is OK. We're done setting up SSL.

The next step is to create the Client Authentication certificate, which IE will pass to IIS to authenticate the user.
makecert -pe -n "CN=Test Client, O=Test company, OU=Testing, E=test@test.com, C=ZA" -ss my -sr CurrentUser -a sha1 -sky exchange -eku 1.3.6.1.5.5.7.3.2 -in "Dev Root Auth" -is Root -ir LocalMachine -sp "Microsoft RSA SChannel Cryptographic Provider" -sy 12 testclient.cer
Note this time the Extended Key Usage value is 1.3.6.1.5.5.7.3.2 which indicates that this cert is for Client Authentication.

The Subject string (-n parameter) must conform to the X.500 distinguished names standard. See RFC2253 for a description of the possible X.500 attributes. You can also use E to specify an email address. I found that the UID attribute doesn't work with makecert, which is too bad, because it would be nice to be able to assign a custom field to the cert to easily identify the user.

You can now configure the SecureTest application in IIS to require Certificate Authentication. In IIS manager, right-click on the SecureTest directory and to Properties -> Directory Security -> Edit. Click "Require secure channel (SSL)". Click "Require client certificates". Click OK and OK again.

Now open https://localhost/SecureTest/default.aspx in IE again. This time the output should show:
Cert present: True
Subject: C=ZA, E=test@test.com, OU=Testing, O=Test company, CN=Test Client
This verifies the Certificate Authentication works.

To make this more useful, you would use the certificate ("what you have") in your Login screen in addition to (or instead of) a username/password ("what you know") for a two-factor authentication scheme. You could map the client certificate to a Windows user in IIS, or write custom authentication code (e.g. Forms Authentication) using the certificate information to uniquely identify the user... (Although some might argue that using a certificate doesn't mean it's a two-factor scheme).

Remember to use WFetch during the whole process to test everything - it's much easier troubleshooting HTTP issues (e.g. to see the HTTP request and response headers and status codes) than just testing with IE.