The blog of flying mind

Also see: Why should I be optimistic about Trolltech and Nokia?

First, an explanation of my odd choice of terms. I’m using OLE as an umbrella which includes the following pieces of technology:

COM – the fundamental object model, like IUnknown and IClassFactory

DCOM – remoting of COM using IDL, NDR pickling and the SCM

Automation – IDispatch, VARIANT, Type Libraries, etc.

Active/X – Protocols for controls and their containers

Next, some disclaimers:

I am not and have never been a GUI programmer. So anything I know about Windows messages and pumping is from debugging GUI applications, not from writing them. I’m not going to talk about WM_PENCTL notifications or anything else that requires UI knowledge.

Multisoft Group: Custom Software Development and Consulting Service.

Also, I’m going to point out a number of problems with OLE and apartments. The history of the CLR and OLE are closely related. In fact, at one point COM+ 1.0 was known internally as COM98 and the CLR was known internally as COM99. We had some pretty aggressive ship targets back then!

In general, I love OLE and the folks who work on it. Although it is inappropriate for the Internet, DCOM is still the fastest and most enterprise-ready distributed object system out there. In a few ways the architecture of.NET Remoting is superior to DCOM, but we never had the time or resources to even approach the engineering effort that has gone into DCOM. Presumably Indigo will eventually change this situation. I also love COM’s strict separation of contract from implementation, the ability to negotiate for contracts, and so much more.

Help Desk Software: Next generation of Live Chat. Jabber/XMPP Live Chat Server for a website.

Also see: Fix ReturnUrl When Sharing Forms Authentication with Multiple Web Applications

Also see: Memory Model

Also see: Brad Abrams’ pixel8 Interview Podcast posted

Also see: DevWeek 2008 Cross Platform Silverlight Demos

Also see: Eriskay: a Programming Language Based on Game Semantics

Also see: Versioning/Deploying Unmanaged Files

Also see: Channel 9 Interview

Also see: Mix 08 Sessions Published

The bottom line is that OLE has had at least as much impact on Microsoft products and the industry, in its day, as.NET is having now.

But, like anything else, OLE has some flaws. In contrast to the stark architectural beauty of COM and DCOM, late-bound Automation is messy. At the time this was all rolled out to the world, I was at Borland and then Oracle. As an outsider, it was hard for me to understand how one team could have produced such a strange combination.

Of course, Automation has been immensely successful – more successful than COM and DCOM. My aesthetic taste is clearly no predictor of what people want. Generally, people want whatever gets the job done, even if it does so in an ad hoc way. And Automation has enabled an incredible number of application scenarios.

Softwre Development for small and middle size companies. World-class software applications.

Also see: Applied Metamodelling: A Foundation for Language Driven Development

Also see: Avoid DevPath

Also see: TransparentProxy

Also see: DevWeek 2008 Cross Platform Silverlight Demos

Also see: IDE Day in Genoa, Italy

Also see: Should “Membership Stores” Be Permitted in Redmond’s Manufacturing Park Zone?

Apartments

If there’s another part of OLE that I dislike, it’s Single Threaded Apartments. Presumably everyone knows that OLE offers three kinds of apartments:

Single Threaded Apartment (STA) – one affinitized thread is used to call all the objects residing in the apartment. Any call on these objects from other threads must perform cross-thread marshaling to this affinitized thread, which dispatches the call. Although a process can have an arbitrary number of STAs (with a corresponding number of threads), most client processes have a single Main STA and the GUI thread is the affinitized thread that owns it.

Live Help Server: Jerry Messenger is Jabber/XMPP Live Chat Server for a website.

Also see: Quaker votes

Also see: A VS.NET Macro to Generate Machine Keys.

Also see: IDE Day in Genoa, Italy

Also see: LoadFrom’s Second Bind

Also see: A Quick Fix for the Validator SetFocusOnError Bug

Also see: Big in Japan

Also see: Load(AssemblyName)

Also see: IDE Day in Genoa, Italy

Also see: Microformats are like RFID tags for the Web

Also see: LINQ - The Uber FindControl

Multiple Threaded Apartment (MTA) – each process has at most one MTA at a time. If the current MTA is not being used, OLE may tear it down. A different MTA will be created as necessary later. Most people think of the MTA as not having thread affinity. But strictly speaking it has affinity to a group of threads. This group is the set of all the threads that are not affinitized to STAs. Some of the threads in this group are explicitly placed in the MTA by calling CoInitializeEx. Other threads in this group are implicitly in the MTA because the MTA exists and because these threads haven’t been explicitly placed into STAs. So, by the strict rules of OLE, it is not legal for STA threads to call on any objects in the MTA. Instead, such calls must be marshaled from the calling STA thread over to one of the threads in the MTA before the call can legally proceed.

Developing Customer Relationship Management Solutions. Web, e-Commerce, Database Design and Software Development.

Also see: LINQ - The Uber FindControl

Also see: Java design, operator overloading and people

Also see: Mix 08 Sessions Published

Also see: The Exception Model

Also see: SourceGear at SD West next week

Also see: From C# to Java: Part 3

Also see: Channel 9 Interview

Also see: IDE Day in Genoa, Italy

Also see: Introducing Microsoft Tagspace

Also see: IDE Day in Genoa, Italy

Also see: 2,433 Unread Emails, I feel your pain..

Also see: Avoid DevPath

Also see: Merry Christmas Indeed!

Neutral Apartment (NA) – this is a recent invention (Win2000, I think). There is one NA in the process. Objects contained in the NA can be called from any thread in the process (STA or MTA threads). There are no threads associated with the NA, which is why it isn’t called NTA. Calls into NA objects can be relatively efficient because no thread marshaling is ever required. However, these cross-apartment calls still require a proxy to handle the transition between apartments. Calls from an object in the NA to an object in an STA or the MTA might require thread marshaling. This depends on whether or not the current thread is suitable for calling into the target object. For example, a call from an STA object to an NA object and from there to an MTA object will require thread marshaling during the transition out of the NA into the MTA.

Threading

The MTA is effectively a free-threaded model. (It’s not quite a free-threaded model, because STA threads aren’t strictly allowed to call on MTA objects directly). From an efficiency point of view, it is the best threading model. Also, it imposes the least semantics on the application, which is also desirable. The main drawback with the MTA is that humans can’t reliably write free-threaded code.

Well, a few developers can write this kind of code if you pay them lots of money and you don’t ask them to write very much. And if you code review it very carefully. And you test it with thousands of machine hours, under very stressful conditions, on high-end MP machines like 8-ways and up. And you’re still prepared to chase down a few embarrassing race conditions once you’ve shipped your product.

But it’s not a good plan for the rest of us.

The NA model is truly free-threaded, in the sense that any thread in the process can call on these objects. All such threads must still transition through a proxy layer that maintains the apartment boundary. But within the NA all calls are direct and free-threaded. This is the only apartment that doesn’t involve thread affinity.

Although the NA is free-threaded, it is often used in conjunction with a lock to achieve rental threading. The rental model says that only one thread at a time can be active inside an object or a group of objects, but there is no restriction on which thread this might be. This is efficient because it avoids thread marshaling. Rather than marshaling a call from one thread to whatever thread is affinitized to the target objects, the calling thread simply acquires the lock (to rent the context) and then completes the call on the current thread. When the thread returns back out of the context, it releases the lock and now other threads can make calls.

If you call out of a rental context into some other object (as opposed to the return pathway), you have a choice. You can keep holding the lock, in which case other threads cannot rent the context until you fully unwind. In this mode, the rental context supports recursion of the current thread, but it does not support reentrancy from other threads. Alternatively, the thread could release the lock when it calls out of the rental context, in which case it must reacquire the lock when it unwinds back and returns to the rental context. In this mode, the rental context supports full reentrancy.

Throughout this blog, we’ll be returning to this fundamental decision of whether to support reentrancy. It’s a complex issue.

If only recursion is supported on a rental model, it’s clear that this is a much more forgiving world for developers than a free-threaded model. Once a thread has acquired the rental lock, no other threads can be active in the rented objects until the lock has been released. And the lock will not be released until the thread fully unwinds from the call into the context.

Even with reentrancy, the number of places where concurrency can occur is limited. Unless the renting thread calls out of the context, the lock won’t be released and the developer knows that other threads aren’t active within the rented objects. Unfortunately, it might be hard for the developer to know all the places that call out of the current context, releasing the lock. Particularly in a componentized world, or a world that combines application code with frameworks code, the developer can rarely have sufficient global knowledge.

So it sounds like limiting a rental context to same-thread recursion is better than allowing reentrancy during call outs, because the developer doesn’t have to worry about other threads mutating the state of objects in the rental context. This is true. But it also means that the resulting application is subject to more deadlocks. Imagine what can happen if two rental contexts are simultaneously making calls to each other. Thread T1 holds the lock to rent context C1. Thread T2 holds the lock to rent context C2. If T1 calls into C2 just as T2 calls into C1, and we are on the recursion plan, we have a classic deadlock. Two locks have been taken in different sequences by two different threads. Alternatively, if we are on a reentrancy plan, T1 will release the lock for C1 before contending for the lock on C2. And T2 will release the lock for C2 before contending for the lock on C1. The deadlock has been avoided, but T1 will find that the objects in C1 have been modified when it returns. And T2 will find similar surprises when it returns to C2.

Affinity

Anyway, we now understand the free-threaded model of the MTA and NA and we understand how to build a rental model on top of these via a lock. How about the single-threaded affinitized model of STAs? It’s hard to completely describe the semantics of an STA, because the complete description must incorporate the details of pages of OLE pumping code, the behavior of 3^rd party IMessageFilters, etc. But generally an STA can be thought of as an affinitized rental context with reentrancy and strict stacking. By this I mean:

It is affinitized rental because all calls into the STA must marshal to the correct thread and because only one logical call can be active in the objects of the apartment at any time. (This is necessarily the case, since there is only ever one thread).
It has reentrancy because every callout from the STA thread effectively releases the lock held by the logical caller and allows other logical callers to either enter or return back to the STA.
It has strict stacking because one stack (the stack of the affinitized STA thread) is used to process all the logical calls that occur in the STA. When these logical calls perform a callout, the STA thread reentrantly picks up another call in, and this pushes the STA stack deeper. When the first callout wants to return to the STA, it must wait for the STA thread’s stack to pop all the way back to the point of its own callout.

That point about strict stacking is a key difference between true rental and the affinitized rental model of an STA. With true rental, we never marshal calls between threads. Since each call occurs on its own thread, the pieces of stack for different logical threads are never mingled on an affinitized thread’s actual stack. Returns back into the rental context after a callout can be processed in any order. Returns back into an STA after a callout must be processed in a highly constrained order.

We’ve already seen a number of problems with STAs due to thread affinity, and we can add some more. Here’s the combined list:

Marshaling calls between threads is expensive, compared to taking a lock.

Processing returns from callouts in a constrained fashion can lead to inefficiencies. For instance, if the topmost return isn’t ready for processing yet, should the affinitized thread favor picking up a new incoming call (possibly leading to unconstrained stack growth) or should it favor waiting for the topmost return to complete (possibly idling the affinitized thread completely and conceivably resulting in deadlocks).

Any conventional locks held by an affinitized thread are worthless. The affinitized thread is processing an arbitrary number of logical calls, but a conventional lock (like an OS CRITICAL_SECTION or managed Monitor) will not distinguish between these logical calls. Instead, all lock acquisitions are performed by the single affinitized thread and are granted immediately as recursive acquisitions. If you are thinking of building a more sophisticated lock that avoids this issue, realize that you are making that classic reentrancy vs. deadlock decision all over again.

Imagine a common server situation. The first call comes in from a particular client, creates a few objects (e.g. a shopping cart) and returns. Subsequent calls from that client manipulate that initial set of objects (e.g. putting some items into the shopping cart). A final call checks out the shopping cart, places the order, and all the objects are garbage collected. Now imagine that all those objects are affinitized to a particular thread. As a consequence, the dispatch logic of your server must ensure that all calls from the same client are routed to the same thread. And if that thread is busy doing other work, the dispatch logic must delay processing the new request until the appropriate affinitized thread is available. This is complicated and it has a severe impact on scalability.

STAs must pump. (How did I get this far without mentioning pumping?)

Any STA code that assumed a single-threaded world for the process, rather than just for the apartment, might not pump. Such code breaks when we introduce the CLR into the process, as we will see.

Failure to Pump

Let’s look at those last two bullet points in more detail. When your STA thread is doing nothing else, it needs to be checking to see if any other threads want to marshal some calls into it. This is done with a Windows message pump. If the STA thread fails to pump, these incoming calls will be blocked. If the incoming calls are GUI SendMessages or PostMessages (which I think of as synchronous or asynchronous calls respectively), then failure to pump will produce an unresponsive UI. If the incoming calls are COM calls, then failure to pump will result in calls timing out or deadlocking.

If processing one incoming call is going to take a while, it may be necessary to break up that processing with intermittent visits to the message pump. Of course, if you pump you are allowing reentrancy to occur at those points. So the developer loses all his wonderful guarantees of single threading.

Unfortunately, there’s a whole lot of STA code out there which doesn’t pump adequately. For the most part, we see this in non-GUI applications. If you have a GUI application that isn’t pumping enough, it’s obvious right there on the screen. Those bugs tend to get fixed.

For non-GUI applications, a failure to pump may not be noticed in unmanaged code. When that code is moved to managed (perhaps by re-compiling some VB6 code as VB.NET), we start seeing bugs. Let’s look at a couple of real-world cases that we encountered during V1 of the CLR and how the lingering effects of these cases are still causing major headaches for managed developers and for Microsoft Support. I’ll describe a server case first, and then a client case.

< ?xml:namespace prefix = st1 ns = "urn:schemas-microsoft-com:office:smarttags" />ADO and ASP Compatibility Mode

ADO.NET and ASP.NET are a winning combination. But ASP.NET also supports an ASP compatibility mode. In this mode, legacy ASP pages can be served up by the managed ASP.NET pipeline. Such pages were written before we invented our managed platform, so they use ADO rather than ADO.NET for any data access. Also, in this mode the DCOM threadpool is used rather than the managed System.Threading.ThreadPool. Although all the threads in the managed ThreadPool are explicitly placed in the MTA (as you might hope and expect), the DCOM threadpool actually contains STA threads.

The purpose of this STA threadpool was to allow legacy STA COM objects in general, and VB6 objects in particular, to be moved from the client to the server. The result suffers from the scaling problems I alluded to before, since requests are dispatched on up to 100 STA threads with careful respect for any affinity. Also, VB6 has a variable scope which corresponds to “global” (I forget its name), but which is treated as per-thread when running on the server. If there are more than 100 clients using a server, multiple clients will share a single STA thread based on the whims of the request dispatch logic. This means that global variables are shared between sets of clients in a surprising fashion, based on the STA that they happen to correspond to.

A typical ASP page written in VBScript would establish a (hopefully pooled) database connection from ADO, query up a row, modify a field, and write the row back to the database. Since the page was likely written in VB, any COM AddRef and Release calls on the ADO row and field value objects were supplied through the magic of the VB6 runtime. This means they occur on the same thread and in a very deterministic fashion.

The ASP page contains no explicit pumping code. Indeed, at no point was the STA actually pumped. Although this is a strict violation of the rules, it didn’t cause any problems. That’s because there are no GUI messages or inter-apartment COM calls that need to be serviced.

This technique of executing ASP pages on STAs with ADO worked fairly well – until we tried to extend the model to ASP.NET running in ASP compatibility mode. The first problem that we ran into was that all managed applications are automatically multi-threaded. For any application of reasonable complexity, there are sure to be at least a few finalizable objects. These objects will have their Finalize methods called by one or more dedicated finalizer threads that are distinct from the application threads.

(It’s important that finalization occurs on non-application threads, since we don’t want to be holding any application locks when we call the Finalize method. And today the CLR only has a single Finalizer thread, but this is an implementation detail. It’s quite likely that in the future we will concurrently call Finalize methods on many objects, perhaps by moving finalization duties over to the ThreadPool. This would address some scalability concerns with finalization, and would also allow us to make stronger guarantees about the availability of the finalization service).

Our COM Interop layer ensures that we almost only ever call COM objects in the correct apartment and context. The one place where we violate COM rules is when the COM object’s apartment or context has been torn down. In that case, we will still call IUnknown::Release on the pUnk to try to recover its resources, even though this is strictly illegal. We’ve gone backwards and forwards on whether this is appropriate, and we provide a Customer Debug Probe so that you can detect whether this is happening in your application.

Anyway, let’s pretend that we absolutely always call the pUnk in the correct apartment and context. In the case of an object living in an STA, this means that the Finalizer thread will marshal the call to the affinitized thread of that STA. But if that STA thread is not pumping, the Finalizer thread will block indefinitely while attempting to perform the cross-thread marshaling.

The effect on a server is crippling. The Finalizer thread makes no progress. The number of unreleased pUnks grows without bounds. Eventually some resource (usually memory) is exceeded and the process crashes.

One solution is to edit the original ASP page to pump the underlying STA thread that it is executing on. A light-weight way to pump is to call Thread.CurrentThread.Join(0). This causes the current thread to block until the current thread dies (which isn’t going to happen) or until 0 milliseconds have elapsed – whichever happens first. I’ll explain later why this also performs some pumping and why this is a controversial aspect of the CLR. A heavier-weight way to pump is to call GC.WaitForPendingFinalizers. This not only performs pumping, but it also waits for the Finalization queue to drain.

If you are porting a page that produces a modest number of COM objects, doing a simple Join on each page may be sufficient. If your page performs elaborate processing, perhaps creating an unbounded number of COM objects in a loop, then you may need to either add a Join within the loop or WaitForPendingFinalizers at the end of the page processing. The only way to really know is to experiment with both techniques, measuring the growth of the Finalization queue and the impact on server throughput.

ADO’s Threading Model

There was another problem with using ADO from ASP.NET’s ASP compatibility mode. Do you know what the threading model of ADO is? Well, if you check the registry for some ADO CLSIDs on your machine, you may find them registered as ThreadingModel=Single or you may find them registered as ThreadingModel=Both.

If these classes are registered as Single, OLE will carefully ensure that their instances can only be called from the thread that they were created on. This implies that the objects can assume a single-threaded view of the world and they do not need to be written in a thread-safe manner. If these classes are registered as Both, OLE will ensure that their instances are only called from threads in the right apartment. But if that apartment is the MTA, these objects better have been written in a thread-safe manner. For example, they had better be using InterlockedIncrement and Decrement, or an equivalent, for reference counting.

Unfortunately, the ADO classes are not thread-safe. Strictly speaking, they should never be registered as anything but Single. You may find them registered as Both on your machine because this improves scalability and throughput for some key scenarios. And those key scenarios happen to limit themselves to “one thread at a time” because of how ASP and VB6 work.

In fact, the legacy ADO classes don’t even support single-threaded access if there is reentrancy. They will randomly crash when used in this manner (and this is exactly the manner in which ADO was driven in the early days of V1). Here are the steps:

The page queries up an ADO row object, which enters managed code via COM Interop as an RCW (runtime-callable wrapper).
By making a COM call on this RCW, the page navigates to a field value. This field value also enters managed code via COM Interop as an RCW.
The page now makes a COM call via ADO which results in a call out to the remote database. At this point, the STA thread is pumped by the DCOM remote call. Since this is a remote call, it’s going to take a while before it returns.
The garbage collector decides that it’s time to collect. At this point, the RCW for the field value is still reachable and is reported. The RCW for the row object is no longer referenced by managed code and is collected.
The Finalizer thread notices that the pUnk underlying the row’s RCW is no longer in use, and it makes the cross-apartment call from the Finalizer thread’s apartment (MTA) to the ADO row object’s apartment (STA).
Recall that the STA thread is pumping for the duration of the remote database call (#3 above). It picks up the cross-thread call from the Finalizer (#5 above) and performs the Release on the Row object. This is the final Release and ADO deletes the unmanaged Row object from memory. This logical call unwinds and the Finalizer thread is unblocked (hurray). The STA thread returns to pumping.
The remote database call returns back to the server machine. The STA thread picks it up from its pumping loop and returns back to the page, unwinding the thread.
The page now updates the field value, which involves a COM call to the underlying ADO object.
ADO crashes or randomly corrupts memory.

What happened? The ADO developers made a questionable design decision when they implemented COM reference counting throughout their hierarchy. The field values refer to their owning row object, but they don’t hold a reference count on that row. Instead, they assume that the row will live as long as all of its associated field values. And yet, whenever the application makes an ADO call on a field value, the field value will access that (hopefully present) row.

This assumption worked fine in the days of ASP and VB6. So nobody even noticed the bug until the CLR violated those threading assumptions – without violating the underlying OLE rules, of course.

It was impractical to fix this by opening up ADO and rewriting the code. There are many different versions of ADO in existence, and many products that distribute it. Another option was to add GC.KeepAlive(row) calls at the bottom of each page, to extend the lifetime of the row objects until the field values were no longer needed. This would have been a nightmare for Support.

Instead, the ADO team solved the problem for managed code with a very elegant technique. (I invented it, so of course I think it was elegant). They opened up the assembly that was created by TlbImp’ing ADO. Then they added managed references from the RCWs of the field values to the RCWs of their owning rows. These managed references are completely visible to the garbage collector. Now the GC knows that if the field values are reachable then the row values must also be reachable. Problem solved.

No Typelib Registered

Incidentally, we ran into another very common problem when we moved existing client or server COM applications over to managed code. Whenever an application uses a COM object, it tries hard to match the thread of the client to the ThreadingModel of the server. In other words, if the application needs to use a ThreadingModel=Main COM object, the application tries to ensure that the creating thread is in an STA. Similarly, if the application needs to use a ThreadingModel=Free COM object, it tries to create this object from an MTA thread. Even if a COM object is ThreadingModel=Both, the application will try to access the object from the same sort of thread (STA vs. MTA) as the thread that created the object.

One reason for doing this is performance. If you can avoid an apartment transition, your calls will be much faster. Another reason has to do with pumping and reentrancy. If you make a cross-apartment call into an STA, the STA better be pumping to pick up your call. And if you make a cross-apartment call out of an STA, your thread will start pumping and your application becomes reentrant. This is a small dose of free-threading, and many application assumptions start to break. A final reason for avoiding apartment transitions is that they often aren’t supported. For instance, most ActiveX scenarios require that the container and the control are in the same STA. If you introduce an apartment boundary (even between two STAs), bizarre cases like Input Synchronous messages stop working properly.

The net result is that a great many applications avoid using COM objects across apartment boundaries. And this means that – even if that COM object is nominally marshalable across an apartment boundary – this often isn’t being tested. So an application might install itself without ensuring that the typelib of the COM component is actually registered.

When the application is moved to managed code, developers are frustrated to see InvalidCastExceptions on the managed side. A typical sequence is that they successfully ‘new’ the COM object, implying that the CoCreate returned a pUnk which was wrapped in an RCW. Then when they cast it to one of the interfaces that they know is supported, a casting exception is thrown. This casting exception is due to a QueryInterface call failing with E_NOINTERFACE. Yet this HRESULT is not returned by the COM object, which does indeed support the interface. Instead, it is returned by a COM apartment proxy which sits between the RCW and that COM object. The COM apartment proxy is simply failing to marshal the interface across the apartment boundary – usually because the COM object is using the OLEAUT marshaler and the Typelib has not been properly registered.

This is a common failure, and it’s unfortunate that a generic E_NOINTERFACE doesn’t lead to better debuggability for this case.

Finally, I can’t help but mention that the COM Interop layer added other perturbations to many unmanaged COM scenarios that seemed to be working just fine. Common perturbations from managed code include garbage collection, a Finalizer thread, strict conformance to OLE marshaling rules, and the fact that managed objects are agile with respect to COM apartments and COM+ contexts (unless they derive from ServicedComponent).

For instance, Trident required that all calls on its objects occur on the correct thread. But Trident also had an extension model where 3^rd party objects could be aggregated onto their base objects. Unfortunately, the aggregator performed blind delegation to the 3^rd party objects. And – even more unfortunate – this blind delegation did not exclude QI’s for IMarshal. Of course, managed objects implement IMarshal to achieve their apartment and context agility. So if Trident aggregated a managed object as an extention, the containing Trident object would attempt to become partially agile in a very broken way.

Hopefully we found and dealt with most of these issues before we shipped V1.

Not Pumping a Client

I said I would describe two cases where non-pumping unmanaged code caused problems when we moved to managed code. The above explains, in great detail, how ADO and ASP compatibility mode caused us problems on the server. Now let’s look at the non-GUI client case.

We all know that a WinForms GUI client is going to put the main GUI thread into an STA. And we know that there’s a lot of pumping in a GUI application, or else not much is going to show on the screen.

Assume for a moment that a Console application also puts its main thread into an STA. If that main thread creates any COM objects via COM Interop, and if those COM objects are ThreadingModel=Main or Both, then the application better be pumping. If it fails to pump, we’ll have exactly the same situation with our server running ASP compatibility mode. The Finalizer thread won’t be able to marshal calls into the STA to Release any pUnks.

On a well-loaded server, that failure is quickly noticed by the developer or by the folks in operations. But on a client, this might be just a mild case of constipation. The rate of creation of finalizable objects may be low enough that the problem is never noticed. Or it may be noticed as a gradual build up of resources. If the problem is reported to Microsoft Support, the customer generally categorizes it as a garbage collection bug.

So what is the apartment of a Console application’s main thread? Well, it depends.

If you build a Console application in Notepad, the main thread is likely to start off in the MTA. If you build a Console application with Visual Studio, then if you pick C# or VB.NET your main thread is likely to be in an STA. If you build a Console application with Visual Studio and you choose managed C++, your main thread is likely to be in an MTA for V1 or V1.1. I think it’s likely to be in an STA for our next release.

Wow. Why are we all over the place on this? Mostly, it’s because there is no correct answer. Either the developer is not going to use any COM objects in his Console application, in which case the choice doesn’t really matter, or the developer is going to use some COM objects and this should inform his decision.

For instance, if the developer will use COM objects with ThreadingModel=Main, he probably wants to put his main thread into an STA so he can use the COM objects directly without cross-thread marshaling and all the issues that this would imply. This means he should also pump that thread, if there are other threads (like the Finalizer!) active in the process. Alternatively, if the developer intends to use COM objects with ThreadingModel=Free, he probably wants to put his main thread in the MTA so he can access those objects directly. Now he doesn’t need to pump, but he does need to consider the implications of writing free-threaded code.

Either way, the developer has some responsibility.

Unfortunately, the choice of a default is typically made by the project type that he selects in Visual Studio, or is based on the CLR’s default behavior (which favors MTA). And realistically the subtleties of apartments and pumping are beyond the knowledge (or interest) of most managed developers. Let’s face it: nobody should have to worry about this sort of thing.

The Managed CoInitialize Mess

There are three ways to select an apartment choice for the main thread of your Console application. All three of these techniques have concerns associated with them.

1) You can place either an STAThreadAttribute or MTAThreadAttribute onto the main method.

2) You can perform an assignment to System.Threading.CurrentThread.ApartmentState as one of the first statements of your main method (or of your thread procedure if you do a Thread.Start).

3) You can accept the CLR’s default of MTA.

So what’s wrong with each of these techniques?

The first technique is the preferred method, and it works very well for C#. After some tweaks to the VB.NET compiler before we shipped V1, it worked well for VB too. Managed C++ still doesn’t properly support this technique. The reason is that the entrypoint of a managed C++ EXE isn’t actually your ‘main’ routine. Instead, it’s a method inside the C-runtime library. That method eventually delegates to your ‘main’ routine. But the CLR doesn’t scan through the closure of calls from the entrypoint when looking for the custom attribute that defines the threading model. If the CLR doesn’t find it on the method that is the EXE’s entrypoint, it stops looking. The net result is that your attribute is quietly ignored for C++.

I’m told that this will be addressed in Whidbey, by having the linker propagate the attribute from ‘main’ to the CRT entrypoint. And indeed this is how the VB.NET compiler works today.

What’s wrong with the second technique? Unfortunately, it is subject to a race condition. Before the CLR can actually call your thread procedure, it may first call some module constructors, class constructors, AssemblyLoad notifications and AssemblyResolve notifications. All of this execution occurs on the thread that was just created. What happens if some of these methods set the thread’s ApartmentState before you get a chance? What happens if they call Windows services like the clipboard that also set the apartment state? A more likely scenario is that one of these other methods will make a PInvoke call that marshals a BSTR, SAFEARRAY or VARIANT. Even these innocuous operations can force a CoInitializeEx on your thread and limit your ability to configure the thread from your thread procedure.

When you are developing your application, none of the above is likely to occur. The real nightmare scenario is that a future version of the CLR will provide a JIT that inlines a little more aggressively, so some extra class constructors execute before your thread procedure. In other words, you will ship an application that is balanced on a knife edge here, and this will become an App Compatibility issue for all of us. (See http://blogs.msdn.com/cbrumme/archive/2003/11/10/51554.aspx for more details on the sort of thing we worry about here).

In fact, for the next release of the CLR we are seriously considering making it impossible to set the apartment state on a running thread in this manner. At a minimum, you should expect to see a Customer Debug Probe warning of the risk here.

And the third technique from above has a similar problem. Recall that threads in the MTA can be explicitly placed there through a CoInitializeEx call, or they can be implicitly treated as being in the MTA because they haven’t been placed into an STA. The difference between these two cases is significant.

If a thread is explicitly in the MTA, any attempt to configure it as an STA thread will fail with an error of RPC_E_CHANGED_MODE. By contrast, if a thread is implicitly in the MTA it can be moved to an STA by calling CoInitializeEx. This is more likely than it may sound. If you attempt a clipboard operation, or you call any number of other Windows services, the code you call may attempt to place your thread in the STA. And when you accept the CLR default behavior, it currently leaves the thread implicitly in the MTA and therefore is subject to reassignment.

This is another place where we are seriously considering changing the rules in the next version of the CLR. Rather than place threads implicitly in the MTA, we are considering making this assignment explicit and preventing any subsequent reassignment. Once again, our motivation is to reduce the App Compat risk for applications after they have been deployed.

Speaking of race conditions and apartments, the CLR has a nasty bug which was introduced in V1 and which we have yet to remove. I’ve already mentioned that any threads that aren’t in STAs or explicitly in the MTA are implicitly in the MTA. That’s not strictly true. These threads are only in the MTA if there is an MTA for them to be in.

There is an MTA if OLE is active in the process and if at least one thread is explicitly in the MTA. When this is the case, all the other unconfigured threads are implicitly in the MTA. But if that one explicit thread should terminate or CoUninitialize, then OLE will tear down the MTA. A different MTA may be created later, when a thread explicitly places itself into it. And at that point, all the unconfigured threads will implicitly join it.

But this destruction and recreation of the MTA has some serious impacts on COM Interop. In fact, any changes to the apartment state of a thread can confuse our COM Interop layer, cause deadlocks on downlevel platforms, and lead to memory leaks and violation of OLE rules.

Let’s look at how this specific race condition occurs first, and then I’ll talk about the larger problems here.

An unmanaged thread CoInitializes itself for the MTA and calls into managed code.
While in managed code, that thread introduces some COM objects to our COM Interop layer in the form of RCWs, perhaps by ‘new’ing them from managed code.
The CLR notices that the current thread is in the MTA, and realizes that it must “keep the MTA alive.” We signal the Finalizer thread to put itself explicitly into the MTA via CoInitializeEx.
The unmanaged thread returns out to unmanaged code where it either dies or simply calls CoUninitialize. The MTA is torn down.
The Finalizer thread wakes up and explicitly CoInitializes itself into the MTA. Oops. It’s too late to keep the original MTA alive and it has the effect of creating a new MTA. At least this one will live until the end of the process.

As far as I know, this is the only race condition in the CLR that we haven’t fixed. Why have we ignored it all these years? First, we’ve never seen it reported from the field. This isn’t so surprising when you consider that the application often shares responsibility for keeping the MTA alive. Many applications are aware of this obligation and – if they use COM – they always keep an outstanding CoInitialize on one MTA thread so the apartment won’t be torn down. Second, I generally resist fixing bugs by adding inter-thread dependencies. It would be all too easy to create a deadlock by making step 3 wait for the Finalizer thread to CoInitialize itself, rather than just signaling it to do so. This is particularly true since the causality of calls from the Finalizer to other threads is often opaque to us, as I’ll explain later. And we certainly don’t want to create a dedicated thread for this purpose. Dedicated threads have a real impact on Terminal Server scenarios, where the cost of one thread in a process is multiplied by all the processes that are running. Even if we were prepared to pay this cost, we would want to create this thread lazily. But synchronizing with the creation of another thread is always a dangerous proposition. Thread creation involves taking the OS loader lock and making DLL_THREAD_ATTACH notifications to all the DllMain routines that didn’t explicitly disable these calls.

The bottom line is that the fix is expensive and distasteful. And it speaks to a more general problem, where many different components in a process may be individually spinning up threads to keep the MTA from being recycled. A better solution is for OLE to provide an API to keep this apartment alive, without requiring all those dedicated threads. This is the approach that we are pursuing for the long term.

In our general cleanup of the CLR’s treatment of CoInitialize, we are also likely to change the semantics of assigning the current thread’s ApartmentState to Unknown. In V1 & V1.1 of the CLR, any attempt to set the state to Unknown would throw an ArgumentOutOfRangeException, so we’re confident that we can make this change without breaking applications.

If the CLR has performed an outstanding CoInitializeEx on this thread, we may treat the assignment to Unknown as a request to perform a CoUninitialize to reverse the operation. Currently, the only way you can CoUninitialize a thread is to PInvoke to the OLE32 service. And such changes to the apartment state are uncoordinated with the CLR.

Now why does it matter if the apartment state of a thread changes, without the CLR knowing? It matters because:

1) The CLR may hold RCWs over COM objects in the apartment that is about to disappear. Without a notification, we cannot legally release those pUnks. As I’ve already mentioned, we break the rules here and attempt to Release anyway. But it’s still a very bad situation and sometimes we will end up leaking.

2) The CLR will perform limited pumping of STA threads when you perform managed blocking (e.g. WaitHandle.WaitOne). If we are on a recent OS, we can use the IComThreadingInfo interface to efficiently determine whether we should pump or not. But if we are on a downlevel platform, we would have to call CoInitialize prior to each blocking operation and check for a failure code to absolutely determine the current state of the thread. This is totally impractical from a performance point of view. So instead we cache what we believe is the correct apartment state of the thread. If the application performs a CoInitialize or CoUninitialize without informing us, then our cached knowledge is stale. So on downlevel platforms we might neglect to pump an STA (which can cause deadlocks). Or we may attempt to pump an MTA (which can cause deadlocks).

Incidentally, if you ever run managed applications under a diagnostic tool like AppVerifier, you may see complaints from that tool at process shutdown that we have leaked one or more CoInitialize calls. In a well-behaved application, each CoInitialize would have a balancing CoUninitialize. However, most processes are not so well-behaved. It’s typical for applications to terminate the process without unwinding all the threads of the process. There’s a very detailed description of the CLR’s shutdown behavior at http://blogs.msdn.com/cbrumme/archive/2003/08/20/51504.aspx .

The bottom line here is that the CLR is heavily dependent on knowing exactly when apartments are created and destroyed, or when threads become associated or disassociated with those apartments. But the CLR is largely out of the loop when these operations occur, unless they occur through managed APIs. Unfortunately, we are rarely informed. For an extreme example of this, the Shell has APIs which require an STA. If the calling thread is implicitly in the MTA, these Shell APIs CoInitialize that calling thread into an STA. As the call returns, the API will CoUnitialize and rip down the apartment.

We would like to do better here over time. But there are some pretty deep problems and most solutions end up breaking an important scenario here or there.

Back to Pumping

Enough of the CoInitialize mess. I mentioned above that managed blocking will perform some pumping when called on an STA thread.

Managed blocking includes a contentious Monitor.Enter, WaitHandle.WaitOne, WaitHandle.WaitAny, GC.WaitForPendingFinalizers, our ReaderWriterLock and Thread.Join. It also includes anything else in FX that calls down to these routines. One noticeable place where this happens is during COM Interop. There are pathways through COM Interop where a cache miss occurs on finding an appropriate pUnk to dispatch a call. At those points, the COM call is forced down a slow path and we use this as an opportunity to pump a little bit. We do this to allow the Finalizer thread to release any pUnks on the current STA, if the application is neglecting to pump. (Remember those ASP Compat and Console client scenarios?) This is a questionable practice on our part. It causes reentrancy at a place where it normally could never occur in pure unmanaged scenarios. But it allows a number of applications to successfully run without clogging up the Finalizer thread.

Anyway, managed blocking does not include PInvokes directly to any of the OS blocking services. And keep in mind that if you PInvoke to the OS blocking services directly, the CLR will no longer be able to take control of your thread. Operations like Thread.Interrupt, Thread.Abort and AppDomain.Unload will be indefinitely delayed.

Did you notice that I neglected to mention WaitHandle.WaitAll in the list of managed blocking opeprations? That’s because we don’t allow you to call WaitAll from an STA thread. The reason is rather subtle. When you perform a pumping wait, at some level you need to call MsgWaitForMultipleObjectsEx, or a similar Msg* based variant. But the semantics of a WAIT_ALL on an OS MsgWaitForMultipleObjectsEx call is rather surprising and not what you want at all. It waits for all the handles to be signaled AND for a message to arrive at the message queue. In other words, all your handles could be signaled and the application will keep blocking until you nudge the mouse! Ugh.

We’ve toyed with some workarounds for this case. For example, you could imagine spinning up an MTA thread and having it perform the blocking operation on the handles. When all the handles are signaled, it could set another event. The STA thread would do a WaitHandle.WaitOne on that other event. This gives us the desired behavior that the STA thread wakes up when all handles are signaled, and it still pumps the message queue. However, if any of those handles are “thread-owned”, like a Mutex, then we have broken the semantics. Our sacrificial MTA thread now owns the Mutex, rather than the STA thread.

Another technique would be to put the STA thread into a loop. Each iteration would ping the handles with a brief timeout to see if it could acquire them. Then it would check the message queue with a PeekMessage or similar technique, and then iterate. This is a terrible solution for battery-powered devices or for Terminal Server scenarios. What used to be efficient blocking is now busily spinning in a loop. And if no messages actually arrive, we have disturbed the fairness guarantees of the OS blocking primitives by pinging.

A final technique would be to acquire the handles one by one, using WaitOne. This is probably the worst approach of all. The semantics of an OS WAIT_ALL are that you will either get no handles or you will get all of them. This is critical to avoiding deadlocks, if different parts of the application block on the same set of handles – but fill the array of handles in random order.

I keep saying that managed blocking will perform “some pumping” when called on an STA thread. Wouldn’t it be great to know exactly what will get pumped? Unfortunately, pumping is a black art which is beyond mortal comprehension. On Win2000 and up, we simply delegate to OLE32’s CoWaitForMultipleHandles service. And before we wrote the initial cut of our pumping code for NT4 and Win9X, I thought I would glance through CoWaitForMultipleHandles to see how it is done. It is many, many pages of complex code. And it uses special flags and APIs that aren’t even available on Win9X.

The code we finally wrote for the downlevel platforms is relatively simple. We gather the list of hidden OLE windows associated with the current STA thread and try to restrict our pumping to the COM calls which travel through them. However, a lot of the pumping complexity is in USER32 services like PeekMessage. Did you know that calling PeekMessage for one window will actually cause SendMessages to be dispatched on other windows belonging to the same thread? This is another example of how someone made a tradeoff between reentrancy and deadlocks. In this case, the tradeoff was made in favor of reentrancy by someone inside USER32.

By now you may be thinking “Okay. Pump more and I get reentrancy. Pump less and I get deadlocks.” But of course the world is more complicated than that. For instance, the Finalizer thread may synchronously call into the main GUI STA thread, perhaps to release a pUnk there, as we have seen. The causality from the Finalizer thread to the main GUI STA thread is invisible to the CLR (though the CLR Security Lead recently suggested using OLE channel hooks as a technique for making this causality visible). If the main GUI STA thread now calls GC.WaitForPendingFinalizers in order to pump, there’s a possibility of a deadlock. That’s because the GUI STA thread must wait for the Finalizer thread to drain its queue. But the Finalizer thread cannot drain its queue until the GUI thread has serviced its incoming synchronous call from the Finalizer.

Reentrancy, Avalon, Longhorn and the Client

Ah, reentrancy again. From time to time, customers inside or outside the company discover that we are pumping messages during managed blocking on an STA. This is a legitimate concern, because they know that it’s very hard to write code that’s robust in the face of reentrancy. In fact, one internal team completely avoids managed blocking, including almost any use of FX, for this reason.

Avalon was very upset, too. I’m not sure how much detail they have disclosed about their threading model. And it’s certainly not my place to reveal what they are doing. Suffice it to say that their model is an explicit rental model that does not presume thread affinity. If you’ve read this far, I’m sure you approve of their decision.

Avalon must necessarily coexist with STAs, but Avalon doesn’t want to require them. The CLR and Avalon have a shared long term goal of driving STAs out of the platform. But, realistically, this will take decades. Avalon’s shorter term goal is to allow some useful GUI applications to be written without STAs. Even this is quite difficult. If you call the clipboard today, you will have an STA.

Avalon also has made a conscious design choice to favor deadlocks over reentrancy. In my opinion, this is an excellent goal. Deadlocks are easily debugged. Reentrancy is almost impossible to debug. Instead, it results in odd inconsistencies that manifest over time.

In order to achieve their design goals, Avalon requires the ability to control the CLR’s pumping. And since we’ve had similar requests from other teams inside and outside the company, this is a reasonable feature for us to provide.

V1 of the CLR had a conscious goal of making as much legacy VB and C++ code work as was possible. When we saw the number of applications that failed to pump, we had no choice but to insert pumping for them – even at the cost of reentrancy. Avalon is in a completely different position. All Avalon code is new code. They are in a great position to define an explicit model for pumping, and then require that all new applications rigorously conform to that model.

Indeed, as much as I dislike STAs, I have a bigger concern about Longhorn and its client focus. Historically, Microsoft has built a ton of great functionality and added it to the platform. But that functionality is often mixed up with various client assumptions. STAs are probably the biggest of those assumptions. The Shell is an example of this. It started out as a user-focused set of services, like the namespace. But it’s growing into something that’s far more generally useful. To the extent that the Shell wants to take its core concepts and make them part of the base managed Longhorn platform, it needs to shed the client focus. The same is true of Office.

For instance, I want to write some code that navigates to a particular document through some namespace and then processes it in some manner. And I want that exact same code to run correctly on the client and on the server. On the client, my processing of that document should not make the UI unresponsive. On the server, my processing of that document should not cause problems with scalability or throughput.

Historically, this just hasn’t been the case. We have an opportunity to correct this problem once, with the major rearchitecture that is Longhorn. But although Longhorn will have both client and server releases, I worry that we might still have a dangerous emphasis on the client.

This may be one of the biggest risks we face in Longhorn.

Winding Down

Finally, I feel a little bad about picking something I don’t like and writing about it. But there’s a reason that this topic came up. Last week, a customer in Japan was struggling with using mshtml.dll to crack some HTML files from inside ASP.NET. It’s the obvious thing to do. Clearly ‘mshtml’ stands for Microsoft HTML and clearly this is how we expect customers to process files in this format.

Unfortunately, MSHTML was written as client-side functionality. In fact, I’m told that it drives its own initialization by posting Windows messages back to itself and waiting for them to be pumped. So if you aren’t pumping an STA, you aren’t going to get very far.

There’s that disturbing historical trend at Microsoft to combine generally useful functionality with a client bias again!

We explained to the customer the risks of using client components on a server, and the pumping behavior that is inherent in managed blocking on an STA. After we had been through all the grisly details, the customer made the natural observation: None of this is written down anywhere.

Well, I still never talked about a mysterious new flag to CoWaitForMultipleHandles. Or how custom implementations of IMessageFilter can cause problems. Or the difference between Main and Single. Or the relationship between apartments and COM+ contexts and ServicedComponents. Or the amazing discovery that OLE32 sometimes requires you to pump the MTA if you have DCOM installed on Win9X.

But I’m sure that at this point I’ve said far more than most people care to hear about this subject.

http://blogs.msdn.com/cbrumme/archive/2004/02/02/66219.aspx

Natural Sorting in C#

Administrator — Tue, 25 Mar 2008 10:48:14 +0000

Also see: Debugging an InvalidCastException

Jeff Atwood recently posted about natural sorting. This is all about making sure that strings that contain numbers sort numerically. I’m slightly surprised to see that he wants to call it alphabetical sorting. Surely by definition, alphabetical sorting is defined by, well, the alphabet. This is an issue about numbers, not letters.

Anyway, he says he tried and gave up on a succinct C# version. He suggests that it will take 40+ lines of code. I believe that’s misleading, because as far as I can tell, the Python versions are only able to be so succinct because Python already appears to know how to sort an array. Both examples he shows rely on this. In.NET, collections aren’t intrinsically sortable. Let’s sort that:

/// 
/// Compares two sequences.
/// 
/// Type of item in the sequences.
/// 
/// Compares elements from the two input sequences in turn. If we
/// run out of list before finding unequal elements, then the shorter
/// list is deemed to be the lesser list.
/// 
public class EnumerableComparer : IComparer<IEnumerable>
{
 /// 
 /// Create a sequence comparer using the default comparer for T.
 /// 
 public EnumerableComparer()
 {
 comp = Comparer.Default;
 }
	
 /// 
 /// Create a sequence comparer, using the specified item comparer
 /// for T.
 /// 
 /// Comparer for comparing each pair of
 /// items from the sequences.
 public EnumerableComparer(IComparer comparer)
 {
 comp = comparer;
 }
	
 /// 
 /// Object used for comparing each element.
 /// 
 private IComparer comp;
	
 /// 
 /// Compare two sequences of T.
 /// 
 /// First sequence.
 /// Second sequence.
 public int Compare(IEnumerable x, IEnumerable y)
 {
 using (IEnumerator leftIt = x.GetEnumerator())
 using (IEnumerator rightIt = y.GetEnumerator())
 {
 while (true)
 {
 bool left = leftIt.MoveNext();
 bool right = rightIt.MoveNext();
	
 if (!(left || right)) return 0;
	
 if (!left) return -1;
 if (!right) return 1;
	
 int itemResult = comp.Compare(leftIt.Current, rightIt.Current);
 if (itemResult != 0) return itemResult;
 }
 }
 }
}

(Note: I offer the code samples on this page under the MIT license.)

So yes, I need a lot of code. However, that’s a utility class that is applicable to a wide range of scenarios, not just this one. It’s slightly irritating that it’s not already built into the.NET framework. Heck, maybe it is, and I’ve just been looking in the wrong place.

Also see: Link Love: 09/21/2007

Also see: Sometimes, it’s the small things..

Also see: VS.NET Macro To Group and Sort Your Using Statements

Also see: Link Love: 09/21/2007

Also see: Tagspace, Meet Claimspace

Given easy way to compare two sequences, a C# 3.0 natural sort becomes roughly as trivial as the Python examples in Jeff’s blog:

string[] testItems = { “z24″, “z2″, “z15″, “z1″,
 “z3″, “z20″, “z5″, “z11″,
 “z 21″, “z22″ };
	
Func<string, object> convert = str =>
{ try { return int.Parse(str); }
 catch { return str; } };
var sorted = testItems.OrderBy(
 str => Regex.Split(str.Replace(” “, “”), “([0-9]+)”).Select(convert),
 new EnumerableComparer<object>());

It’s probably not meaningful to count lines of code. This being C#, I could have put it all on one line. As it is, I split it across more lines than I normally would, to avoid an annoying HTML layout issue. (I put my code samples in PRE blocks to get the formatting right, PRE blocks and long lines are a bad combination.) But I think it’s fair to say that any differences in size are due merely to syntactic differences between Python and C#. Structurally, there’s no substantial difference – I’ve been able to apply exactly the same techniques the Python examples used in C#.

Live Person: Live Chat Solution for Online Customer Service on Website.

Also see: DevWeek 2008 Silverlight Precon Demos

Also see: Life Calculus

Also see: The PDC and Application Compatibility, but still no Hosting

Also see: JSR-203 more New I/O APIs - NIO.2

Also see: Java design, operator overloading and people

Also see: Updated Finalization and Hosting

Also see: Resizing a Form has always been a pain in the rectum…

Also see: Snippet Compiler update

Also see: Interested in Artificial Intelligence? What about Wiki’s? Well, now you can have both.

Also see: Sometimes, it’s the small things..

If I print out the results using this code:

foreach (string s in sorted)
{
 Console.WriteLine(s);
}

It prints out the test items in this order:

z1
z2
z3
z5
z11
z15
z20
z 21
z22
z24

I.e., ascending numeric order, rather than what you’d get with most string ordering.

[Updated 21st December 2007: Charles Petzold didn’t like the original version, which treated spaces as significant for sorting. So I’ve updated the example to ignore spaces, as the position of “z 21” in the output above shows. I simply added a call to Replace(" ", "") on the string before passing it into Regex.Split.]

http://www.interact-sw.co.uk/iangblog/2007/12/13/natural-sorting

Updated Finalization and Hosting

Administrator — Tue, 25 Mar 2008 03:00:02 +0000

Also see: Memory Model

My original posts on Finalization and Hosting had some hokey XXXXX markers in place of content, where that content hadn’t already been disclosed in some form. Now that the Visual Studio 2005 Community Preview is available, I’ve gone back to those two posts and replaced the XXXXX markers with real text.

Also, it’s obviously been a while since my last post. I started writing something this weekend, but the weather here has been spectacular and I was compelled to go outside and play. I’ll try to have something in the next couple of weeks.

http://blogs.msdn.com/cbrumme/archive/2004/04/26/120609.aspx

Startup, Shutdown and related matters

Administrator — Mon, 24 Mar 2008 20:00:13 +0000

Also see: Single source code base for Silverlight and WPF solutions

Usually
I write blog articles on topics that people request via email or comments on
other blogs. Well, nobody has ever
asked me to write anything about shutdown.

size=2>

But then
I look at all the problems that occur during process shutdown in the unmanaged
world. These problems occur because
many people don’t understand the rules, or they don’t follow the rules, or the
rules couldn’t possibly work anyway.

size=2>

We’ve
taken a somewhat different approach for managed applications. But I don’t think we’ve ever explained
in detail what that approach is, or how we expect well-written applications to
survive an orderly shutdown.
Furthermore, managed applications still execute within an unmanaged OS
process, so they are still subject to the OS rules. And in V1 and V1.1 of the CLR we’ve
horribly violated some of those OS rules related to startup and shutdown. We’re trying to improve our behavior
here, and I’ll discuss that too.

size=2>

Also see: Brad Abrams’ pixel8 Interview Podcast posted

Also see: A web site is not an RSS feed…nor the reverse.

Also see: The Internet is Officially Dead & Boring - Its the economy stupid !

Also see: TransparentProxy

Also see: A web site is not an RSS feed…nor the reverse.

Also see: Be my Support Group

Also see: Mix 08 Sessions Published

Also see: A web site is not an RSS feed…nor the reverse.

Also see: ASP.NET MVC in CodePlex and Extensible Unit Testing

Questionable
APIs

size=2>Unfortunately, I can’t discuss the model for shutting down managed
applications without first discussing how unmanaged applications terminate. And, as usual, I’ll go off on a bunch of
wild tangents.

size=2>

size=2>Ultimately, every OS process shuts down via a call to ExitProcess or
TerminateProcess. ExitProcess is
the nice orderly shutdown, which notifies each DLL of the termination. TerminateProcess is ruder, in that the
DLLs are not informed.

size=2>

The
relationship between ExitProcess and TerminateProcess has a parallel in the
thread routines ExitThread and TerminateThread. ExitThread is the nice orderly thread
termination, whereas if you ever call TerminateThread you may as well kill the
process. It’s almost guaranteed to
be in a corrupt state. For example,
you may have terminated the thread while it holds the lock for the OS heap. Any thread attempting to allocate or
release memory from that same heap will now block forever.

Multisoft Group: Custom software solutions for your business.

Also see: Mix 08 Sessions Published

Also see: Music and Movies - Give Away the Soundtrack

Also see: Exception Handling in Running a Business

Also see: Quaker votes

Also see: Music and Movies - Give Away the Soundtrack

Also see: Exception Handling in Running a Business

Also see: Link Love: 09/21/2007

Also see: NHL seven days a week

Also see: Microformats are like RFID tags for the Web

size=2>

size=2>Realistically, Win32 shouldn’t contain a TerminateThread service. To a first approximation, anyone who has
ever used this service has injected a giant bug into his application. But it’s too late to remove it
now.

size=2>

In that
sense, TerminateThread is like System.Threading.Thread.Suspend and Resume. I cannot justify why I added those
services. The OS SuspendThread and
ResumeThread are extremely valuable to a tiny subset of applications. The CLR itself uses these routines to
take control of threads for purposes like Garbage Collection and – as we’ll see
later – for process shutdown. As
with TerminateThread, there’s a significant risk of leaving a thread suspended
at a “bad” spot. If you call
SuspendThread while a thread is inside the OS heap lock, you better not try to
allocate or free from that same heap.
In a similar fashion, if you call SuspendThread while a thread holds the
OS loader lock (e.g. while the thread is executing inside DllMain) then you
better not call LoadLibrary, GetProcAddress, GetModuleHandle, or any of the other OS
services that require that same lock.

Live Person: Live Chat Solution for Online Customer Service on Website.

size=2>

Even
worse, if you call SuspendThread on a thread that is in the middle of exception
dispatching inside the kernel, a subsequent GetThreadContext or SetThreadContext
can actually produce a blend of the register state at the point of the
suspension and the register state that was captured when the exception was
triggered. If we attempt to modify
a thread’s context (perhaps bashing the EIP – on X86 – to redirect the thread’s
execution to somewhere it will synchronize with the GC or other managed
suspension), our update to EIP might quietly get lost. Fortunately it’s possible to coordinate
our user-mode exception dispatching with our suspension attempts in order to
tolerate this race condition.

And
probably the biggest gotcha with using the OS SuspendThread & ResumeThread
services is on Win9X. If a Win9X
box contains real-mode device drivers (and yes, some of them still do), then
it’s possible for the hardware interrupt associated with the device to interact
poorly with the thread suspension.
Calls to GetThreadContext can deliver a register state that is perturbed
by the real-mode exception processing.
The CLR installs a VxD on those operating systems to detect this case and
retry the suspension.

Live Person Software: Turn website visitors into your customers.

Also see: Brad Abrams’ pixel8 Interview Podcast posted

Also see: Note to self: Blog about using Service Broker

Also see: Memory Model

size=2>

Anyway,
with sufficient care and discipline it’s possible to use the OS SuspendThread
& ResumeThread to achieve some wonderful things.

size=2>

But the
managed Thread.Suspend & Resume are harder to justify. They differ from the unmanaged
equivalents in that they only ever suspend a thread at a spot inside managed
code that is “safe for a garbage collection.” In other words, we can report all the GC
references at that spot and we can unwind the stack and register state to reveal
our caller’s execution state.

size=2>

Because
we are at a place that’s safe for garbage collection, we can be sure that
Thread.Suspend won’t leave a thread suspended while it holds an OS heap
lock. But it may be suspended while
it holds a managed Monitor (‘lock’ in C# or ‘SyncLock’ in VB.NET). Or it may be suspended while it is
executing the class constructor (.cctor) of an important class like
System.String. And over time we
intend to write more of the CLR in managed code, so we can enjoy all the
benefits. When that happens, a
thread might be suspended while loading a class or resolving security policy for
a shared assembly or generating shared VTables for COM Interop.

Live Help Server: Jerry Messenger Server is Live Chat with Users on your websites.

size=2>

The real
problem is that developers sometimes confuse Thread.Suspend with a
synchronization primitive. It is
not. If you want to synchronize two
threads, you should use appropriate primitives like Monitor.Enter,
Monitor.Wait, or WaitHandle.WaitOne.
Of course, it’s harder to use these primitives because you actually have
to write code that’s executed by both threads so that they cooperate
nicely. And you have to eliminate
the race conditions.

size=2>

I’m
already wandering miles away from Shutdown, and I need to get back. But I can’t resist first mentioning that
TerminateThread is distinctly different from the managed Thread.Abort service,
both in terms of our aspirations and in terms of our current
implementation.

size=2>

Nobody
should ever call TerminateThread.
Ever.

Help Desk Software: for your business. Java Custom Software Soulutions and Service.

Also see: DevWeek 2008 Cross Platform Silverlight Demos

Also see: Mix 08 Sessions Published

Also see: Be my Support Group

Also see: LoadFile vs. LoadFrom

size=2>

Today
you can safely call Thread.Abort in two scenarios.

size=2>

style="MARGIN: 0in 0in 0pt; mso-list: l8 level1 lfo4; tab-stops: list.5in"
>You can call Abort on your own thread
(Thread.CurrentThread.Abort()).
This is not much different than throwing any exception on your thread,
other than the undeniable manner in which the exception propagates. The propagation is undeniable in the
sense that your thread will continue to abort, even if you attempt to swallow
the ThreadAbortException in a catch clause. At the end-catch, the CLR notices that
an abort is in progress and we re-throw the abort. You must either explicitly call the
ResetAbort method – which carries a security demand – or the exception must
propagate completely out of all managed handlers, at which point we reset the
undeniable nature of the abort and allow unmanaged code to (hopefully) swallow
it.

size=2>

style="MARGIN: 0in 0in 0pt; mso-list: l8 level1 lfo4; tab-stops: list.5in"
>An Abort is performed on all threads that have stack in an
AppDomain that is being unloaded.
Since we are throwing away the AppDomain anyway, we can often tolerate
surprising execution of threads at fairly arbitrary spots in their
execution. Even if this leaves
managed locks unreleased and AppDomain statics in an inconsistent state, we’re
throwing away all that state as part of the unload anyway. This situation isn’t as robust as we
would like it to be. So we’re
investing a lot of effort into improving our behavior as part of getting
“squeaky clean” for highly available execution inside SQL Server in our next
release.

Help Desk Software: for your business. Java Custom Software Soulutions and Service.

size=2>

Longer
term, we’re committed to building enough reliability infrastructure around
Thread.Abort that you can reasonably expect to use it to control threads that
remain completely inside managed code.
Aborting threads that interleave managed and unmanaged execution in a
rich way will always remain problematic, because we are limited in how much we
can control the unmanaged portion of that execution.

size=2>

ExitProcess
in a nutshell

So what
does the OS ExitProcess service actually do? I’ve never read the source code. But based on many hours of stress
investigations, it seems to do the following:

size=2>

style="MARGIN: 0in 0in 0pt 0.5in; TEXT-INDENT: -0.25in; mso-list: l0 level1 lfo1; tab-stops: list.5in">1)
Kill all the threads except one,
whatever they are doing in user mode.
On NT-based operating systems, the surviving thread is the thread that
called ExitProcess. This becomes
the shutdown thread. On Win9X-based
operating systems, the surviving thread is somewhat random. I suspect that it’s the last thread to
get around to committing suicide.

Help Desk Software: Next generation of Live Chat. Jabber/XMPP Live Chat Service for your website.

size=2>

style="MARGIN: 0in 0in 0pt 0.5in; TEXT-INDENT: -0.25in; mso-list: l0 level1 lfo1; tab-stops: list.5in">2)
Once only one thread survives, no
further threads can enter the process… almost. On NT-based systems, I only see
superfluous threads during shutdown if a debugger attaches to the process during
this window. On Win9X-based
systems, any threads that were created during this early phase of shutdown are
permitted to start up. The
DLL_THREAD_ATTACH notifications to DllMain for the starting threads will be
arbitrarily interspersed with the DLL_PROCESS_DETACH notifications to DllMain
for the ensuing shutdown. As you
might expect, this can cause crashes.

size=2>

style="MARGIN: 0in 0in 0pt 0.5in; TEXT-INDENT: -0.25in; mso-list: l0 level1 lfo1; tab-stops: list.5in">3)
Since only one thread has survived
(on the more robust NT-based operating systems), the OS now weakens all the
CRITICAL_SECTIONs. This is mixed
blessing. It means that the
shutdown thread can allocate and free objects from the system heap without
deadlocking. And it means that
application data structures protected by application CRITICAL_SECTIONs are
accessible. But it also means that
the shutdown thread can see corrupt application state. If one thread was wacked in step #1
above while it held a CRITICAL_SECTION and left shared data in an inconsistent
state, the shutdown thread will see this inconsistency and must somehow tolerate
it. Also, data structures that are
protected by synchronization primitives other than CRITICAL_SECTION are still
prone to deadlock.

Live Chat (Help Desk) Server Software —
Improve performance of customer support with Jerry Messenger. Live Chat Support to your website visitors.

Also see: Resizing a Form has always been a pain in the rectum…

size=2>

style="MARGIN: 0in 0in 0pt 0.5in; TEXT-INDENT: -0.25in; mso-list: l0 level1 lfo1; tab-stops: list.5in">4)
The OS calls the DllMain of each
loaded DLL, giving it a DLL_PROCESS_DETACH notification. The ‘lpReserved’ argument to DllMain
indicates whether the DLL is being unloaded from a running process or whether
the DLL is being unloaded as part of a process shutdown. (In the case of the CLR’s DllMain, we
only ever receive the latter style of notification. Once we’re loaded into a process, we
won’t be unloaded until the process goes away).

size=2>

style="MARGIN: 0in 0in 0pt 0.5in; TEXT-INDENT: -0.25in; mso-list: l0 level1 lfo1; tab-stops: list.5in">5)
The process actually terminates,
and the OS reclaims all the resources associated with the process.

Help Desk Software: for your business. Java Custom Software Soulutions and Service.

Also see: A web site is not an RSS feed…nor the reverse.

Also see: Merry Christmas Indeed!

Also see: Resizing a Form has always been a pain in the rectum…

Also see: Link Love: 09/21/2007

Also see: Compatability

Also see: Be my Support Group

size=2>

Well,
that sounds orderly enough. But try
running a multi-threaded process that calls ExitProcess from one thread and
calling HeapAlloc / HeapFree in a loop from a second thread. If you have a debugger attached,
eventually you will trap with an ‘INT 3’ instruction in the OS heap code. The OutputDebugString message will
indicate that a block has been freed, but has not been added to the free list…
It has been leaked. That’s because
the ExitProcess wacked your 2^nd thread while it was in the middle of
a HeapFree operation.

size=2>

This is
symptomatic of a larger problem. If
you wack threads while they are performing arbitrary processing, your
application will be left in an arbitrary state. When the DLL_PROCESS_DETACH
notifications reach your DllMain, you must tolerate that arbitrary
state.

size=2>

Softwre Development for small and middle size companies. World-class software applications.

Also see: Resizing a Form has always been a pain in the rectum…

Also see: Note to self: Blog about using Service Broker

Also see: LoadFile vs. LoadFrom

Also see: LoadFrom’s Second Bind

Also see: Memory Model

Also see: Merry Christmas Indeed!

Also see: Access to old blogs

Also see: Big in Japan

Also see: Compatability

I’ve
been told by several OS developers that it is the application’s responsibility
to take control of all the threads before calling ExitProcess. That way, the application will be in a
consistent state when DLL_PROCESS_DETACH notifications occur. If you work in the
operating system, it’s reasonable to consider the “application” to be a
monolithic homogenous piece of code written by a single author. So of course that author should put his
house in order and know what all the threads are doing before calling
ExitProcess.

size=2>

But if
you work on an application, you know that there are always multiple components
written by multiple authors from different vendors. These components are only loosely aware
of each other’s implementations – which is how it should be. And some of these components have extra
threads on the side, or they are performing background processing via
IOCompletion ports, threadpools, or other techniques.

size=2>

Under
those conditions, nobody can have the global knowledge and global control
necessary to call ExitProcess “safely”.
So, regardless of the official rules, ExitProcess will be called while
various threads are performing arbitrary processing.

size=2>

The OS
Loader Lock

It’s
impossible to discuss the Win32 model for shutting down a process without
considering the OS loader lock.
This is a lock that is present on all Windows operating systems. It provides mutual exclusion during
loading and unloading.

size=2>

size=2>Unfortunately, this lock is held while application code executes. This fact alone is sufficient to
guarantee disaster.

size=2>

If you
can avoid it, you must never hold one of your own locks while calling into
someone else’s code. They will
screw you every time.

size=2>

Like all
good rules, this one is made to be broken.
The CLR violates this rule in a few places. For example, we hold a ‘class
constructor’ lock for your class when we call your.cctor method. However, the CLR recognizes that this
fact can lead to deadlocks and other problems. So we have rules for weakening this lock
when we discover cycles of.cctor locks in the application, even if these cycles
are distributed over multiple threads in multi-threaded scenarios. And we can see through various other
locks, like the locks that coordinate JITting, so that larger cycles can be
detected. However, we deliberately
don’t look through user locks (though we could see through many of these, like
Monitors, if we chose). Once we
discover a visible, breakable lock, we allow one thread in the cycle to see
uninitialized state of one of the classes.
This allows forward progress and the application continues. See my earlier blog on “Initializing
code” for more details.

size=2>

size=2>Incidentally, I find it disturbing that there’s often little discipline
in how managed locks like Monitors are used. These locks are so convenient,
particularly when exposed with language constructs like C# lock and VB.NET
SyncLock (which handle backing out of the lock during exceptions), that many
developers ignore good hygiene when using them. For example, if code uses multiple locks
then these locks should typically be ranked so that they are always acquired in
a predictable order. This is one
common technique for avoiding deadlocks.

size=2>

Anyway,
back to the loader lock. The
OS takes this lock implicitly when it is executing inside APIs like
GetProcAddress, GetModuleHandle and GetModuleFileName. By holding this lock inside these APIs,
the OS ensures that DLLs are not loading and unloading while it is groveling
through whatever tables it uses to record the state of the process.

size=2>

So if
you call those APIs, you are implicitly acquiring a lock.

size=2>

That
same lock is also acquired during a LoadLibrary, FreeLibrary, or CreateThread
call. And – while it is held – the
operating system will call your DllMain routine with a notification. The notifications you might see
are:

size=2>

DLL_THREAD_ATTACH

The
thread that calls your DllMain has just been injected into the process. If you need to eagerly allocate any TLS
state, this is your opportunity to do so.
In the managed world, it is preferable to allocate TLS state lazily on
the first TLS access on a given thread.

size=2>

DLL_THREAD_DETACH

The
thread that calls your DllMain has finished executing the thread procedure that
it was started up with. After it
finishes notifying all the DLLs of its death in this manner, it will
terminate. Many unmanaged
applications use this notification to de-allocate their TLS data. In the managed world, managed TLS is
automatically cleaned up without your intervention. This happens as a natural consequence of
garbage collection.

size=2>

DLL_PROCESS_ATTACH

The
thread that calls your DllMain is loading your DLL via an explicit LoadLibraryEx
call or similar technique, like a static bind. The lpReserved argument indicates
whether a dynamic or static bind is in progress. This is your opportunity to initialize
any global state that could not be burned into the image. For example, C++ static initializers
execute at this time. The managed
equivalent has traditionally been a class constructor method, which executes
once per AppDomain. In a future
version of the CLR, we hope to provde a more convenient module constructor
concept.

size=2>

DLL_PROCESS_DETACH

If the
process is terminating in an orderly fashion (ExitProcess), your DllMain will
receive a DLL_PROCESS_DETACH notification where the lpReserved argument is
non-null. If the process is
terminating in a rude fashion (TerminateProcess), your DllMain will receive no
notification. If someone unloads
your DLL via a call to FreeLibrary or equivalent, the process will continue
executing after you unload. This case is indicated by a null value for
lpReserved. In the managed world, de-initialization
happens through notifications of AppDomain unload or process exit, or through
finalization activity.

The DLL_THREAD_ATTACH and
DLL_THREAD_DETACH calls have a performance implication. If you have loaded
100 DLLs into your process and you start a new thread, that thread must call 100
different DllMain routines. Let’s say that these routines touch a page or
two of code each, and a page of data. That might be 250 pages (1 MB) in your
working set, for no good reason.

The CLR calls DisableThreadLibraryCalls
on all managed assemblies other than certain MC++ IJW assemblies (more on this
later) to avoid this overhead for you. And it’s a good idea to do the same on your
unmanaged DLLs if they don’t need these notifications to manage their
TLS.

Writing code inside DllMain is one of
the most dangerous places to write code. This is because you are executing inside a
callback from the OS loader, inside the OS loader lock.

Here are some of the rules related to
code inside DllMain:

style="MARGIN: 0in 0in 0pt 0.5in; TEXT-INDENT: -0.25in; mso-list: l10 level1 lfo2; tab-stops: list.5in">1) You must never call LoadLibrary or
otherwise perform a dynamic bind.

style="MARGIN: 0in 0in 0pt 0.5in; TEXT-INDENT: -0.25in; mso-list: l10 level1 lfo2; tab-stops: list.5in">2) You must never attempt to acquire a
lock, if that lock might be held by a thread that needs the OS loader lock. (Acquiring a heap
lock by calling HeapAlloc or HeapFree is probably okay).

style="MARGIN: 0in 0in 0pt 0.5in; TEXT-INDENT: -0.25in; mso-list: l10 level1 lfo2; tab-stops: list.5in">3) You should never call into another
DLL. The
danger is that the other DLL may not have initialized yet, or it may have
already uninitialized. (Calling into kernel32.dll is probably
okay).

style="MARGIN: 0in 0in 0pt 0.5in; TEXT-INDENT: -0.25in; mso-list: l10 level1 lfo2; tab-stops: list.5in">4) You should never start up a thread or
terminate a thread, and then rendezvous with that other thread’s start or
termination.

As we shall see, the CLR violates some
of these rules.
And these violations have resulted in serious consequences for managed
applications – particularly managed applications written in MC++.

And if you’ve ever written code inside
DllMain – including code that’s implicitly inside DllMain like C++ static
initializers or ‘atexit’ routines – then you’ve probably violated some of these
rules. Rule #3
is especially harsh.

The fact is, programs violate these
rules all the time and get away with it. Knowing this, the MC++ and CLR teams made a
bet that they could violate some of these rules when executing IJW
assemblies. It
turns out that we bet wrong.

I’m going to explain exactly how we
screwed this up with IJW assemblies, but first I need to explain what IJW
assemblies are.

IJW

IJW is how we internally refer to mixed
managed / unmanaged images. If you compile a MC++ assembly with ‘/clr’ in
V1 or V1.1, it almost certainly contains a mixture of managed and unmanaged
constructs.

In future versions, I expect there will
be ways to compile MC++ assemblies with compiler-enforced guarantees that the
image is guaranteed pure managed, or guaranteed pure verifiable managed, or –
ultimately – perhaps even pure verifiable 32-bit / 64-bit neutral managed. In each case, the
compiler will necessarily have to restrict you to smaller and smaller subsets of
the C++ language.
For example, verifiable C++ cannot use arbitrary unmanaged pointers. Instead, it must
restrict itself to managed pointers and references, which are reported to the
garbage collector and which follow certain strict rules. Furthermore, 32-bit
/ 64-bit neutral code cannot consume the declarations strewn through the
windows.h headers, because these pick a word size during compilation.

IJW is an acronym for “It Just Works”
and it reflects the shared goal of the C++ and CLR teams to transparently
compile existing arbitrary C++ programs into IL. I think we did an amazing job of approaching
that goal, but of course not everything “just works.” First, there are a
number of constructs like inline assembly language that cannot be converted to
managed execution.
The C++ compiler, linker and CLR ensure that these methods are left as
unmanaged and that managed callers transparently switch back to unmanaged before
calling them.

So inline X86 assembly language must
necessarily remain in unmanaged code. Some other constructs are currently left in
unmanaged code, though with sufficient effort we could provide managed
equivalents.
These other constructs include setjmp / longjmp, member pointers (like
pointer to virtual method), and a reasonable startup / shutdown story (which is
what this blog article is supposed to be about).

I’m not sure if we ever documented the
constructs that are legal in a pure managed assembly, vs. those constructs which
indicate that the assembly is IJW. Certainly we have a strict definition of this
distinction embedded in our code, because the managed loader considers it when
loading. Some
of the things we consider are:

style="MARGIN: 0in 0in 0pt; mso-list: l2 level1 lfo3; tab-stops: list.5in"
>A pure
managed assembly has exactly one DLL import. > This import is to mscoree.dll’s _CorExeMain
(for an EXE) or _CorDllMain (for a DLL). > The entrypoint of the EXE or DLL must be a
JMP to this import.
This is how we force the runtime to load and get control whenever a
managed assembly is loaded.

style="MARGIN: 0in 0in 0pt; mso-list: l2 level1 lfo3; tab-stops: list.5in"
>A pure
managed assembly can have no DLL exports. > When we bind to pure managed assemblies, it
is always through managed Fusion services, via AssemblyRefs and assembly
identities (ideally with cryptographic strong names).

style="MARGIN: 0in 0in 0pt; mso-list: l2 level1 lfo3; tab-stops: list.5in"
>A pure
managed assembly has exactly one rebasing fixup. This fixup is for
the JMP through the import table that I mentioned above. Unmanaged EXEs
tend to strip all their rebasing fixups, since EXEs are almost guaranteed to
load at their preferred addresses. > However, managed EXEs can be loaded like
DLLs into a running process. > That single fixup is useful for cases where
we want to load via LoadLibraryEx on versions of the operating system that
support this.

style="MARGIN: 0in 0in 0pt; mso-list: l2 level1 lfo3; tab-stops: list.5in"
>A pure
managed assembly has no TLS section and no other exotic constructs that are
legal in arbitrary unmanaged PE files.

Of course, IJW assemblies can have many
imports, exports, fixups, and other constructs. As with pure managed assemblies, the
entrypoint is constrained to be a JMP to mscoree.dll’s _CorExeMain or
_CorDllMain function.
This is the “outer entrypoint”. However, the COM+ header of the PE file has
an optional “inner entrypoint”. Once the CLR has proceeded far enough into
the loading process on a DLL, it will dispatch to this inner entrypoint which
is… your normal DllMain. In V1 and V1.1, this inner entrypoint is
expressed as a token to a managed function. Even if your DllMain is written as an
unmanaged function, we dispatch to a managed function which is defined as a
PInvoke out to the unmanaged function.

Now we can look at the set of rules for
what you can do in a DllMain, and compare it to what the CLR does when it sees
an IJW assembly.
The results aren’t pretty. Remember that inside DllMain:

You must never call LoadLibrary or otherwise perform a
dynamic bind

With normal managed assemblies, this
isn’t a concern.
For example, most pure managed assemblies are loaded through
Assembly.Load or resolution of an AssemblyRef – outside of the OS loader
lock. Even
activation of a managed COM object through OLE32’s CoCreateInstance will
sidestep this issue.
The registry entries for the CLSID always mention mscoree.dll as the
server. A
subkey is consulted by mscoree.dll – inside DllGetClassObject and outside of the
OS loader lock – to determine which version of the runtime to spin up and which
assembly to load.

But IJW assemblies have arbitrary DLL
exports.
Therefore other DLLs, whether unmanaged or themselves IJW, can have
static or dynamic (GetProcAddress) dependencies on an IJW assembly. When the OS loads
the IJW assembly inside the loader lock, the OS further resolves the static
dependency from the IJW assembly to mscoree.dll’s _CorDllMain. Inside _CorDllMain,
we must select an appropriate version of the CLR to initialize in the
process. This
involves calling LoadLibrary on a particular version of mscorwks.dll, violating
our first rule for DllMain.

So what goes wrong when this rule is
violated?
Well, the OS loader has already processed all the DLLs and their imports,
walking the tree of static dependencies and forming a loading plan. It is now executing
on this plan.
Let’s say that the loader’s plan is to first initialize an IJW assembly,
then initialize its dependent mscoree.dll reference, and then initialize
advapi32.dll.
(By ‘initialize’, I mean give that DLL its DLL_PROCESS_ATTACH
notification).
When mscoree.dll decides to LoadLibrary mscorwks.dll, a new loader plan
must be created.
If mscorwks.dll depends on advapi32.dll (and of course it does), we have
a problem. The
OS loader already has advapi32.dll on its pending list. It will initialize
that DLL when it gets far enough into its original loading plan, but not
before.

If mscorwks.dll needs to call some APIs
inside advapi32.dll, it will now be making those calls before advapi32.dll’s
DllMain has been called. This can and does lead to arbitrary
failures. I
personally hear about problems with this every 6 months or so. That’s a pretty low
rate of failure.
But one of those failures was triggered when a healthy application
running on V1 of the CLR was moved to V1.1 of the CLR. Ouch.

You must never attempt to acquire a lock, if that lock
might be held by a thread that needs the OS loader lock

It’s not possible to execute managed
code without potentially acquiring locks on your thread. For example, we may
need to initialize a class that you need access to. If that class isn’t
already initialized in your AppDomain, we will use a.cctor lock to coordinate
initialization.
Along the same lines, if a method requires JIT compilation we will use a
lock to coordinate this. And if your thread allocates a managed
object, it may have to take a lock. (We don’t take a lock on each allocation if
we are executing on a multi-processor machine, for obvious reasons. But eventually your
thread must coordinate with the garbage collector via a lock before it can
proceed with more allocations).

So if you execute managed code inside
the OS loader lock, you are going to contend for a CLR lock. Now consider what
happens if the CLR ever calls GetModuleHandle or GetProcAddress or
GetModuleFileName while it holds one of those other locks. This includes
implicit calls to LoadLibrary / GetProcAddress as we fault in any lazy DLL
imports from the CLR.

Unfortunately, the sequence of lock
acquisition is inverted on the two threads. This yields a classic deadlock.

Once again, this isn’t a concern for
pure managed assemblies. The only way a pure managed assembly can
execute managed code inside the OS loader lock is if some unmanaged code
explicitly calls into it via a marshaled out delegate or via a COM call from its own
DllMain.
That’s a bug in the unmanaged code! But with an IJW assembly, some methods are
managed and some are unmanaged. The compiler, linker and CLR conspire to make
this fact as transparent as possible. But any call from your DllMain (i.e. from
your inner entrypoint) to a method that happened to be emitted as IL will set
you up for this deadlock.

You should
never call into another DLL

It’s really not possible to execute
managed code without making cross-DLL calls. The JIT compiler is in a different DLL from
the ExecutionEngine.
The ExecutionEngine is in a different DLL from your IJW
assembly.

Once again, pure managed assemblies
don’t usually have a problem here. I did run into one case where one of the
Microsoft language compilers was doing a LoadLibrary of mscorlib.dll. This had the side
effect of spinning up the CLR inside the OS loader lock and inflicting all the
usual IJW problems onto the compilation process. Since managed assemblies have no DLL exports,
it’s rare for applications to load them in this manner. In the case of this
language compiler, it was doing so for the obscure purpose of printing a banner
to the console at the start of compilation, telling the user what version of the
CLR it was bound to.
There are much better ways of doing this sort of thing, and none of those
other ways would interfere with the loader lock. This has been corrected.

You should never start up a thread or terminate a thread,
and then rendezvous

This probably doesn’t sound like
something you would do. And yet it’s one of the most common deadlocks
I see with IJW assemblies on V1 and V1.1 of the CLR. The typical stack
trace contains a load of an IJW assembly, usually via a DLL import. This causes
mscoree.dll’s _CorDllMain to get control. Eventually, we notice that the IJW assembly
has been strong name signed, so we call into WinVerifyTrust in
WinTrust.dll.
That API has a perfectly reasonable expectation that it is not inside the
OS loader lock.
It calls into the OS threadpool (not the managed CLR threadpool), which
causes the OS threadpool to lazily initialize itself. Lazy initialization
involves spinning up a waiter thread, and then blocking until that waiter thread
starts executing.

Of course, the new waiter thread must
first deliver DLL_THREAD_ATTACH notifications to any DLLs that expect such
notifications.
And it must obviously obtain the OS loader lock before it can deliver the
first notification.
The result is a deadlock.

So I’ve painted a pretty bleak picture
of all the things that can go wrong with IJW assemblies in V1 and V1.1 of the
CLR. If we had
seen a disturbing rate of failures prior to shipping V1, we would have
reconsidered our position here. But it wasn’t until later that we had enough
external customers running into these difficulties. With the benefits
of perfect hindsight, it is now clear that we screwed up.

Fortunately, much of this is fixable in
our next release.
Until then, there are some painful workarounds that might bring you some
relief. Let’s
look at the ultimate solution first, and then you can see how the workarounds
compare. We
think that the ultimate solution would consist of several parts:

style="MARGIN: 0in 0in 0pt; mso-list: l6 level1 lfo6; tab-stops: list.5in"
>Just
loading an IJW assembly must not spin up a version of the CLR. That’s because
spinning up a version of the CLR necessarily involves a dynamic load, and
we’ve seen that dynamic loads are illegal during loading and initializing of
static DLL dependencies. > Instead, mscoree.dll must perform enough
initialization of the IJW assembly without actually setting up a full
runtime.
This means that all calls into the managed portion of the IJW assembly
must be bashed so that they lazily load a CLR and initialize it on first
call.

style="MARGIN: 0in 0in 0pt; mso-list: l6 level1 lfo6; tab-stops: list.5in"
>Along the
same lines, the inner entrypoint of an IJW assembly must either be omitted or
must be encoded as an unmanaged entrypoint. > Recall that the current file format doesn’t
have a way of representing unmanaged inner entrypoints, since this is always
in the form of a token. > Even if the token refers to an unmanaged
method, we would have to spin up a version of the CLR to interpret that token
for us. So
we’re going to need a tweak to the current file format to enable unmanaged
inner entrypoints.

style="MARGIN: 0in 0in 0pt; mso-list: l6 level1 lfo6; tab-stops: list.5in"
>An
unmanaged inner entrypoint is still a major risk. If that inner
entrypoint calls into managed code, we will trap the call and lazily spin up
the correction version of the CLR. > At that point, you are in exactly the same
situation as if we had left the entrypoint as managed. Ideally,
assembly-level initialization and uninitialization would never happen inside
the OS loader lock.
Instead, they would be replaced with modern managed analogs that are
unrelated to the unmanaged OS loader’s legacy behavior. If you read my
old blog on “Initializing code” at http://blogs.gotdotnet.com/cbrumme/PermaLink.aspx/611cdfb1-2865-4957-9a9c-6e2655879323 , I mention that we’re under some
pressure to add a module-level equivalent of.cctor methods. That mechanism
would make a great replacement for traditional DLL_PROCESS_ATTACH
notifications.
In fact, the CLR has always supported a.cctor method at a global
module scope.
However, the semantics associated with such a method was that it ran
before any access to static members at global module scope. A more useful
semantic for a future version of the CLR would be for such a global.cctor to
execute before any access to members in the containing Module, whether global
or contained in any of the Module’s types.

style="MARGIN: 0in 0in 0pt; mso-list: l6 level1 lfo6; tab-stops: list.5in"
> >The above changes make it possible to avoid execution of
managed code inside the OS loader lock. > But it’s still possible for a naïve or
misbehaved unmanaged application to call a managed service (like a marshaled
out delegate or a managed COM object) from inside DllMain. This final
scenario is not specific to IJW. > All managed execution is at risk to this
kind of abuse.
Ideally, the CLR would be able to detect attempts to enter it while the
loader lock is held, and fail these attempts. > It’s not clear whether such detection /
prevention should be unconditional or whether it should be enabled through a
Customer Debug Probe. >

size=2>If you don’t know what Customer Debug Probes are,
please hunt them down on MSDN. They are a life-saver for debugging certain
difficult problems in managed applications. I would recommend starting with http://www.gotdotnet.com/Community/UserSamples/Details.aspx?SampleGuid=c7b955c7-231a-406c-9fa5-ad09ef3bb37f , and then reading most of Adam Nathan’s
excellent blogs at http://blogs.gotdotnet.com/anathan .

Of the above 4 changes, we’re relatively
confident that the first 3 will happen in the next release. We also
experimented with the 4^th change, but it’s
unlikely that we will make much further progress. A key obstacle is that there is no
OS-approved way that can efficiently detect execution inside the loader
lock. Our hope
is that a future version of the OS would provide such a mechanism.

This is all great. But you have an
application that must run on V1 or V1.1. What options do you have? Fortunately, Scott
Currie has written an excellent article on this very subject. If you build IJW
assemblies, please read it at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dv_vstechart/html/vcconmixeddllloadingproblem.asp .

The Pure Managed
Story

If you code in a language other than
MC++, you’re saying “Enough about IJW and the OS loader lock
already.”

Let’s look at what the CLR does during
process shutdown.
I’ll try not to mention IJW, but I’ll have to keep talking about that
darn loader lock.

From the point of view of a managed
application, there are three types of shutdown:

style="MARGIN: 0in 0in 0pt 0.5in; TEXT-INDENT: -0.25in; mso-list: l5 level1 lfo7; tab-stops: list.5in">1) A shutdown initiated by a call to
TerminateProcess doesn’t involve any further execution of the CLR or managed
code. From our
perspective, the process simply disappears. This is the rudest of all shutdowns, and
neither the CLR developer nor the managed developer has any obligations related
to it.

style="MARGIN: 0in 0in 0pt 0.5in; TEXT-INDENT: -0.25in; mso-list: l5 level1 lfo7; tab-stops: list.5in">2) A shutdown initiated by a direct call to
ExitProcess is an unorderly shutdown from the point of view of the managed
application.
Our first notification of the shutdown is via a DLL_PROCESS_DETACH
notification.
This notification could first be delivered to the DllMain of
mscorwks.dll, mscoree.dll, or any of the managed assemblies that are currently
loaded.
Regardless of which module gets the notification first, it is always
delivered inside the OS loader lock. It is not safe to execute any managed code at
this time. So
the CLR performs a few house-keeping activities and then returns from its
DllMain as quickly as possible. Since no managed code runs, the managed
developer still has no obligations for this type of shutdown.

style="MARGIN: 0in 0in 0pt 0.5in; TEXT-INDENT: -0.25in; mso-list: l5 level1 lfo7; tab-stops: list.5in">3) An orderly managed shutdown gives
managed code an opportunity to execute outside of the OS loader lock, prior to
calling ExitProcess.
There are several ways we can encounter an orderly shutdown. Because we will
execute managed code, including Finalize methods, the managed developer must
consider this case.

Examples of an orderly managed shutdown
include:

style="MARGIN: 0in 0in 0pt 0.5in; TEXT-INDENT: -0.25in; mso-list: l9 level1 lfo8; tab-stops: list.5in">1) Call System.Environment.Exit(). I already mentioned
that some Windows developers have noted that you must not call ExitProcess
unless you first coordinate all your threads… and then they work like mad to
make the uncoordinated case work. For Environment.Exit we are under no
illusions. We
expect you to call it in races from multiple threads at arbitrary times. It’s our job to
somehow make this work.

style="MARGIN: 0in 0in 0pt 0.5in; TEXT-INDENT: -0.25in; mso-list: l9 level1 lfo8; tab-stops: list.5in">2) If a process is launched with a managed
EXE, then the CLR tracks the number of foreground vs. background managed
threads. (See
Thread.IsBackground).
When the number of foreground threads drops to zero, the CLR performs an
orderly shutdown of the process. Note that the distinction between foreground
and background threads serves exactly this purpose and no other
purpose.

style="MARGIN: 0in 0in 0pt 0.5in; TEXT-INDENT: -0.25in; mso-list: l9 level1 lfo8; tab-stops: list.5in">3) Starting with MSVCRT 7.0, an explicit
call to ‘exit()’ or an implicit call to ‘exit()’ due to a return from ‘main()’
can turn into an orderly managed shutdown. The CRT checks to see if mscorwks.dll or
mscoree.dll is in the process (I forget which). If it is resident, then it calls
CorExitProcess to perform an orderly shutdown. Prior to 7.0, the CRT is of course unaware of
the CLR.

style="MARGIN: 0in 0in 0pt 0.5in; TEXT-INDENT: -0.25in; mso-list: l9 level1 lfo8; tab-stops: list.5in">4) Some unmanaged applications are aware of
the CLR’s requirements for an orderly shutdown. An example is devenv.exe, which is the EXE
for Microsoft Visual Studio. Starting with version 7, devenv calls
CoEEShutDownCOM to force all the CLR’s references on COM objects to be
Release()’d.
This at least handles part of the managed shutdown in an orderly
fashion. It’s
been a while since I’ve looked at that code, but I think that ultimately devenv
triggers an orderly managed shutdown through a 2^nd API.

If you are following along with the
Rotor sources, this all leads to an interesting quirk of EEShutDown in
ceemain.cpp.
That method can be called:

style="MARGIN: 0in 0in 0pt; mso-list: l7 level1 lfo9; tab-stops: list.5in"
>0 times, if
someone calls TerminateProcess.
style="MARGIN: 0in 0in 0pt; mso-list: l7 level1 lfo9; tab-stops: list.5in"
>1 time, if
someone initiates an unorderly shutdown via ExitProcess.
style="MARGIN: 0in 0in 0pt; mso-list: l7 level1 lfo9; tab-stops: list.5in"
>2 times, if
we have a single-threaded orderly shutdown. > In this case, the first call is made
outside of the OS loader lock. > Later, we call ExitProcess for the 2^nd half of the shutdown. This causes
EEShutDown to be called a 2^nd time.
style="MARGIN: 0in 0in 0pt; mso-list: l7 level1 lfo9; tab-stops: list.5in"
>Even more
times, if we have a multi-threaded orderly shutdown. Many threads will
race to call EEShutDown the first time, outside the OS loader lock. This routine
protects itself by anointing a winner to proceed with the shutdown. Then the eventual
call to ExitProcess causes the OS to kill all threads except one, which calls
back to EEShutDown inside the OS loader lock.

Of course, our passage through
EEShutDown is quite different when we are outside the OS loader lock, compared
to when we are inside it. When we are outside, we do something like
this:

style="MARGIN: 0in 0in 0pt; mso-list: l3 level1 lfo10; tab-stops: list.5in"
>First we
synchronize at the top of EEShutDown, to handle the case where multiple
threads race via calls to Environment.Exit or some equivalent
entrypoint.
style="MARGIN: 0in 0in 0pt; mso-list: l3 level1 lfo10; tab-stops: list.5in"
>Then we
finalize all objects that are unreachable. > This finalization sweep is absolutely
normal and occurs while the rest of the application is still running.
style="MARGIN: 0in 0in 0pt; mso-list: l3 level1 lfo10; tab-stops: list.5in"
>Then we
signal for the finalizer thread to finish its normal activity and participate
in the shutdown.
The first thing it does is raise the AppDomain.ProcessExit event. Once we get past
this point, the system is no longer behaving normally. You could either
listen to this event, or you could poll System.Environment.HasShutdownStarted
to discover this fact. > This can be an important fact to discover
in your Finalize method, because it’s more difficult to write robust
finalization code when we have started finalizing reachable
objects.
It’s no longer possible to depend on WaitHandles like Events, remoting
infrastructure, or other objects. > The other time we can finalize reachable
objects is during an AppDomain unload. > This case can be discovered by listening to
the AppDomain.DomainUnload event or by polling for the
AppDomain.IsFinalizingForUnload state. > The other nasty thing to keep in mind is
that you can only successfully listen to the ProcessExit event from the
Default AppDomain.
This is something of a bug and I think we would like to try fixing it
for the next release.
style="MARGIN: 0in 0in 0pt; mso-list: l3 level1 lfo10; tab-stops: list.5in"
>Before we
can start finalizing reachable objects, we suspend all managed activity. This is a
suspension from which we will never resume. > Our goal is to minimize the number of
threads that are surprised by the finalization of reachable state, like static
fields, and it’s similar to how we prevent entry to a doomed AppDomain when we
are unloading it.
style="MARGIN: 0in 0in 0pt; mso-list: l3 level1 lfo10; tab-stops: list.5in"
>This
suspension is unusual in that we allow the finalizer thread to bypass the
suspension.
Also, we change suspended threads that are in STAs, so that they pump
COM messages.
We would never do this during a garbage collection, since the
reentrancy would be catastrophic. > (Threads are suspended for a GC at pretty
arbitrary places… down to an arbitrary machine code instruction boundary in
many typical scenarios). > But since we are never going to resume from
this suspension, and since we don’t want cross-apartment COM activity to
deadlock the shutdown attempt, pumping makes sense here. This suspension
is also unusual in how we raise the barrier against managed execution. For normal GC
suspensions, threads attempting to call from unmanaged to managed code would
block until the GC completes. > In the case of a shutdown, this could cause
deadlocks when it is combined with cross-thread causality (like synchronous
cross-apartment calls). > Therefore the barrier behaves differently
during shutdown.
Returns into managed code block normally. But calls into
managed code are failed. > If the call-in attempt is on an HRESULT
plan, we return an HRESULT. > If it is on an exception plan, we
throw. The
exception code we raise is 0xC0020001 and the argument to RaiseException is a
failure HRESULT formed from the ERROR_PROCESS_ABORTED SCODE (0x1067).
style="MARGIN: 0in 0in 0pt; mso-list: l3 level1 lfo10; tab-stops: list.5in"
>Once all
objects have been finalized, even if they are reachable, then we Release() all
the COM pUnks that we are holding. > Normally, releasing a chain of pUnks from a
traced environment like the CLR involves multiple garbage collections. Each collection
discovers a pUnk in the chain and subsequently Release’s it. If that Release
on the unmanaged side is the final release, then the unmanaged pUnk will be
free’d. If
that pUnk contains references to managed objects, those references will now be
dropped. A
subsequent GC may now collect this managed object and the cycle begins
again. So a
chain of pUnks that interleaves managed and unmanaged execution can require a
GC for each interleaving before the entire chain is recovered. During shutdown,
we bypass all this.
Just as we finalize objects that are reachable, we also drop all
references to unmanaged pUnks, even if they are reachable.

From the perspective of managed code, at
this point we are finished with the shutdown, though of course we perform many
more steps for the unmanaged part of the shutdown.

There are a couple of points to note
with the above steps.

style="MARGIN: 0in 0in 0pt; mso-list: l4 level1 lfo11; tab-stops: list.5in"
>We never
unwind threads.
Every so often developers express their surprise that ‘catch’, ‘fault’,
‘filter’ and ‘finally’ clauses haven’t executed throughout all their threads
as part of a shutdown. > But we would be nuts to try this. It’s just too
disruptive to throw exceptions through threads to unwind them, unless we have
a compelling reason to do so (like AppDomain.Unload). And if those
threads contain unmanaged execution on their threads, the likelihood of
success is even lower. > If we were on that plan, some small
percentage of attempted shutdowns would end up with “Unhandled Exception /
Debugger Attach” dialogs, for no good reason.

style="MARGIN: 0in 0in 0pt; mso-list: l4 level1 lfo11; tab-stops: list.5in"
>Along the
same lines, developers sometimes express their surprise that all the
AppDomains aren’t unloaded before the process exits. Once again, the
benefits don’t justify the risk or the overhead of taking these extra
steps. If
you have termination code you must run, the ProcessExit event and Finalizable
objects should be sufficient for doing so.

style="MARGIN: 0in 0in 0pt; mso-list: l4 level1 lfo11; tab-stops: list.5in"
>We run most
of the above shutdown under the protection of a watchdog thread. By this I mean
that the shutdown thread signals the finalizer thread to perform most of the
above steps.
Then the shutdown thread enters a wait with a timeout. If the timeout
triggers before the finalizer thread has completed the next stage of the
managed shutdown, the shutdown thread wakes up and skips the rest of the
managed part of the shutdown. > It does this by calling ExitProcess. This is almost
fool-proof.
Unfortunately, if the shutdown thread is an STA thread it will pump COM
messages (and SendMessages), while it is performing this watchdog blocking
operation.
If it picks up a COM call into its STA that deadlocks, then the process
will hang.
In a future release, we can fix this by using an extra thread. We’ve hesitated
to do so in the past because the deadlock is exceedingly rare, and because
it’s so wasteful to burn a thread in this manner.

Finally, a lot more happens inside
EEShutDown than the orderly managed steps listed above. We have some
unmanaged shutdown that doesn’t directly impact managed execution. Even here we try
hard to limit how much we do, particularly if we’re inside the OS loader
lock. If we
must shutdown inside the OS loader lock, we mostly just flush any logs we are
writing and detach from trusted services like the profiler or
debugger.

One thing we do not do during
shutdown is any form of leak detection. This is somewhat controversial. There are a number
of project teams at Microsoft which require a clean leak detection run whenever
they shutdown.
And that sort of approach to leak detection has been formalized in
services like MSVCRT’s _CrtDumpMemoryLeaks, for external use. The basic idea is
that if you can find what you have allocated and release it, then you never
really leaked it.
Conversely, if you cannot release it by the time you return from your
DllMain then it’s a leak.

I’m not a big fan of that approach to
finding memory leaks, for a number of reasons:

style="MARGIN: 0in 0in 0pt; mso-list: l11 level1 lfo12; tab-stops: list.5in"
>The fact
that you can reclaim memory doesn’t mean that you were productively using
it. For
example, the CLR makes extensive use of “loader heaps” that grow without
release until an AppDomain unloads. > At that point, we discard the entire heap
without regard for the fine-grained allocations within it. The fact that we
remembered where all the heaps are doesn’t really say anything about whether
we leaked individual allocations within those heaps.
style="MARGIN: 0in 0in 0pt; mso-list: l11 level1 lfo12; tab-stops: list.5in"
>In a few
well-bounded cases, we intentionally leak. > For example, we often build little snippets
of machine code dynamically. > These snippets are used to glue together
pieces of JITted code, or to check security, or twiddle the calling
convention, or various other reasons. > If the circumstances of creation are rare
enough, we might not even synchronize threads that are building these
snippets.
Instead, we might use a light-weight atomic compare/exchange
instruction to install the snippet. > Losing the race means we must discard the
extra snippet.
But if the snippet is small enough, the race is unlikely enough, and
the leak is bounded enough (e.g. we only need one such snippet per AppDomain
or process and reclaim it when the AppDomain or process terminates), then
leaking is perfectly reasonable. > In that case, we may have allocated the
snippet in a heap that doesn’t support free’ing.
style="MARGIN: 0in 0in 0pt; mso-list: l11 level1 lfo12; tab-stops: list.5in"
>This
approach certainly encourages a lot of messy code inside the
DLL_PROCESS_DETACH notification – which we all know is a very dangerous place
to write code.
This is particularly true, given the way threads are wacked by the OS
at arbitrary points of execution. > Sure, all the OS CRITICAL_SECTIONs have
been weakened.
But all the other synchronization primitives are still owned by those
wacked threads.
And the weakened OS critical sections were supposed to protect data
structures that are now in an inconsistent state. If your shutdown
code wades into this landmine of deadlocks and trashed state, it will have a
hard time cleanly releasing memory blocks. > Projects often deal with this case by
keeping a count of all locks that are held. > If this count is non-zero when we get our
DLL_PROCESS_DETACH notification, it isn’t safe to perform leak detection. But this leads to
concerns about how often the leak detection code is actually executed. For a while, we
considered it a test case failure if we shut down a process while holding a
lock. But
that was an insane requirement that was often violated in race
conditions.
style="MARGIN: 0in 0in 0pt; mso-list: l11 level1 lfo12; tab-stops: list.5in"
>The OS is
about to reclaim all resources associated with this process. The OS will
perform a faster and more perfect job of this than the application ever
could. From
a product perspective, leak detection at product shutdown is about the least
interesting time to discover leaks.
style="MARGIN: 0in 0in 0pt; mso-list: l11 level1 lfo12; tab-stops: list.5in"
> >DLL_PROCESS_DETACH notifications are delivered to
different DLLs in a rather arbitrary order. > I’ve seen DLLs either depend on brittle
ordering, or I’ve seen them make cross-DLL calls out of their DllMain in an
attempt to gain control over this ordering. > This is all bad practice. However, I must
admit that in V1 of the CLR, fusion.dll & mscorwks.dll played this “dance
of death” to coordinate their termination. > Today, we’ve moved the Fusion code into
mscorwks.dll.
style="MARGIN: 0in 0in 0pt; mso-list: l11 level1 lfo12; tab-stops: list.5in"
>I think
it’s too easy for developers to confuse all the discipline surrounding this
approach with actually being leak-free. > The approach is so onerous that the goal
quickly turns into satisfying the requirements rather than chasing
leaks.

There are at least two other ways to
track leaks.

One way is to identify scenarios that
can be repeated, and then monitor for leaks during the steady-state of repeating
those scenarios.
For example, we have a test harness which can create an AppDomain, load
an application into it, run it, unload the AppDomain, then rinse and
repeat. The
first few times that we cycle through this operation, memory consumption
increases.
That’s because we actually JIT code and allocate data structures to
support creating a 2^nd AppDomain, or support
making remote calls into the 2^nd AppDomain, or
support unloading that AppDomain. More subtly, the ThreadPool might create –
and retain – a waiter thread or an IO thread. Or the application may trigger the creation
of a new segment in the GC heap which the GC decides to retain even after the
incremental contents have become garbage. This might happen because the GC decides it
is not productive to perform a compacting collection at this time. Even the OS heap
can make decisions about thread-relative look-aside lists or lazy VirtualFree
calls.

But if you ignore the first 5 cycles of
the application, and take a broad enough view over the next 20 cycles of the
application, a trend becomes clear. And if you measure over a long enough period,
paltry leaks of 8 or 12 bytes per cycle can be discovered. Indeed, V1 of the
CLR shipped with a leak for a simple application in this test harness that was
either 8 or 12 bytes (I can never remember which). Of that, 4 bytes
was a known leak in our design. It was the data structure that recorded the
IDs of all the AppDomains that had been unloaded. I don’t know if we’ve subsequently addressed
that leak. But
in the larger scheme of things, 8 or 12 bytes is pretty impressive.

Recently, one of our test developers has
started experimenting with leak detection based on tracing of our unmanaged data
structures.
Fortunately, many of these internal data structures are already described
to remote processes, to support out-of-process debugging of the CLR. The idea is that we
can walk out from the list of AppDomains, to the list of assemblies in each one,
to the list of types, to their method tables, method bodies, field descriptors,
etc. If we
cannot reach all the allocated memory blocks through such a walk, then the
unreachable blocks are probably leaks.

Of course, it’s going to be much harder
than it sounds.
We twiddle bits of pointers to save extra state. We point to the
interiors of heap blocks. We burn the addresses of some heap blocks,
like dynamically generated native code snippets, into JITted code and then
otherwise forget about the heap address. So it’s too early to say whether this
approach will give us a sound mechanism for discovering leaks. But it’s certainly
a promising idea and worth pursuing.

Rambling Security
Addendum

Finally, an off-topic note as I close
down:

I haven’t blogged in about a month. That’s because I
spent over 2 weeks (including weekends) on loan from the CLR team to the DCOM
team. If
you’ve watched the tech news at all during the last month, you can guess
why. It’s
security.

From outside the company, it’s easy to
see all these public mistakes and take a very frustrated attitude. “When will
Microsoft take security seriously and clean up their act?” I certainly
understand that frustration. And none of you want to hear me whine about
how it’s unfair.

The company performed a much publicized
and hugely expensive security push. Tons of bugs were filed and fixed. More importantly,
the attitude of developers, PMs, testers and management was fundamentally
changed.
Nobody on our team discusses new features without considering security
issues, like building threat models. Security penetration testing is a fundamental
part of a test plan.

Microsoft has made some pretty strong
claims about the improved security of our products as a result of these
changes. And
then the DCOM issues come to light.

Unfortunately, it’s still going to be a
long time before all our code is as clean as it needs to be.

Some of the code we reviewed in the DCOM
stack had comments about DGROUP consolidation (remember that precious 64KB
segment prior to 32-bit flat mode?) and OS/2 2.0 changes. Some of these
source files contain comments from the ‘80s. I thought that Win95 was ancient!

I’ve only been at Microsoft for 6
years. But
I’ve been watching this company closely for a lot longer, first as a customer at
Xerox and then for over a decade as a competitor at Borland and Oracle. For the greatest
part of Microsoft’s history, the development teams have been focused on enabling
as many scenarios as possible for their customers. It’s only been for
the last few years that we’ve all realized that many scenarios should never be
enabled. And
many of the remainder should be disabled by default and require an explicit
action to opt in.

One way you can see this change in the
company’s attitude is how we ship products. The default installation is increasingly
impoverished.
It takes an explicit act to enable fundamental goodies, like
IIS.

Another hard piece of evidence that
shows the company’s change is the level of resource that it is throwing at the
problem.
Microsoft has been aggressively hiring security experts. Many are in a new
Security Business Unit, and the rest are sprinkled through the product
groups. Not
surprisingly, the CLR has its own security development, PM, test and penetration
teams.

I certainly wasn’t the only senior
resource sucked away from his normal duties because of the DCOM alerts. Various folks from
the Developer Division and Windows were handed over for an extended period. One of the other
CLR architects was called back from vacation for this purpose.

We all know that Microsoft will remain a
prime target for hacking. There’s a reason that everyone attacks
Microsoft rather than Apple or Novell. This just means that we have to do a lot
better.

Unfortunately, this stuff is still way
too difficult.
It’s a simple fact that only a small percentage of developers can write
thread-safe free-threaded code. And they can only do it part of the
time. The
state of the art for writing 100% secure code requires that same sort of
super-human attention to detail. And a hacker only needs to find a single
exploitable vulnerability.

I do think that managed code can avoid
many of the security pitfalls waiting in unmanaged code. Buffer overruns are
far less likely.
Our strong-name binding can guarantee that you call who you think you are
calling.
Verifiable type safety and automatic lifetime management eliminate a
large number of vulnerabilities that can often be used to mount security
attacks.
Consideration of the entire managed stack makes simple luring attacks
less likely.
Automatic flow of stack evidence prevents simple asynchronous luring
attacks from succeeding. And so on.

But it’s still way too
hard. Looking
forwards, a couple of points are clear:

style="MARGIN: 0in 0in 0pt 0.5in; TEXT-INDENT: -0.25in; mso-list: l1 level1 lfo5; tab-stops: list.5in">1) We need to focus harder on the goal that
managed applications are secure, right out of the box. This means
aggressively chasing the weaknesses of our present system, like the fact that
locally installed assemblies by default run with FullTrust throughout their
execution. It
also means static and dynamic tools to check for security holes.

style="MARGIN: 0in 0in 0pt 0.5in; TEXT-INDENT: -0.25in; mso-list: l1 level1 lfo5; tab-stops: list.5in">2) No matter what we do, hackers will find
weak spots and attack them. The very best we can hope for is that we can
make those attacks rarer and less effective.

I’ll add managed security to my list for
future articles.

http://blogs.msdn.com/cbrumme/archive/2003/08/20/51504.aspx

ReflectionTypeLoadException

Administrator — Mon, 24 Mar 2008 19:48:14 +0000

Also see: Memory Model

Also see: Brad Abrams’ pixel8 Interview Podcast posted

If a type can’t be loaded for some reason during a call to Module.GetTypes(), ReflectionTypeLoadException will be thrown. Assembly.GetTypes() also throws this because it calls Module.GetTypes().

The Message for this exception is “One or more exceptions have been thrown while loading the types” or “Unable to load one or more of the types in the assembly”, which doesn’t seem very descriptive. But, the exception actually provides more info that that. Just get the LoaderExceptions property of the ReflectionTypeLoadException instance. That will give an array of the exceptions caught while loading all of the types from the module. If the exceptions are due to an assembly loading problem, see my general debugging advice.

http://blogs.msdn.com/suzcook/archive/2003/08/11/57236.aspx

Hosting

Administrator — Mon, 24 Mar 2008 19:00:12 +0000

Also see: LINQ - The Uber FindControl

Hosting

< ?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" />

My prior three blogs were supposed to be on Hosting. Each time I got side tracked, first on Exceptions, then on Application Compatibility and finally on Finalization. I refuse to be side tracked this time… much.

Also, I need to explain why it’s taken so long to get this blog out. Part of the reason is vacation. I spent Thanksgiving skiing in Whistler. Then I took a quick side trip to < ?xml:namespace prefix = st1 ns = "urn:schemas-microsoft-com:office:smarttags" />Scottsdale for a friend’s surprise birthday party and to visit my parents. Finally, I spent over three weeks on Maui getting a break from the Seattle winter.

Another reason for the delay is writer’s block. This topic is so huge. The internal specification for the Whidbey Hosting Interfaces is over 100 pages. And that spec only covers the hosting interfaces themselves. There are many other aspects of hosting, like how to configure different security policy in different AppDomains, or how to use COM or managed C++ to stitch together the unmanaged host with the managed applications. There’s no way I can cover the entire landscape.

Also see: Java design, operator overloading and people

Also see: Memory Model

Also see: Big in Japan

Also see: LoadFrom’s Second Bind

Also see: Merry Christmas Indeed!

Also see: Big in Japan

Also see: DevWeek 2008 Cross Platform Silverlight Demos

Also see: Note to self: Blog about using Service Broker

Also see: Access to old blogs

Also see: Resizing a Form has always been a pain in the rectum…

Also see: Java design, operator overloading and people

Also see: Music and Movies - Give Away the Soundtrack

Also see: Compatability

Anyway, here goes.

Mostly I was tourist overhead at the PDC. But one of the places I tried to pay for my ticket was a panel on Hosting. The other panelists included a couple of Program Managers from the CLR, another CLR architect, representatives from Avalon / Internet Explorer, SQL Server, Visual Studio / Office, and – to my great pleasure – a representative from IBM for DB2.

One thing that was very clear at that panel is that the CLR team has done a poor job of defining what hosting is and how it is done. Depending on your definition, hosting could be:

Mixing unmanaged and managed code in the same process.

Running multiple applications, each in its own specially configured AppDomain.

Live Chat Software: Next generation of Live Chat. On-Demand. Easy-to-Use.

Using the unmanaged hosting interfaces described in mscoree.idl.

Configuring how the CLR runs in the process, like disabling the concurrent GC through an application config file.

Even though the hosting interfaces described in mscoree.idl are a small part of what could be hosting, I’m going to concentrate on those interfaces.

In V1 and V1.1 of the CLR, we provided some APIs that allowed an unmanaged process host to exercise some limited control over the CLR. This limited control included the ability to select the version of the CLR to load, the ability to create and configure AppDomains from unmanaged code, access to the ThreadPool, and a few other fundamental operations.

Custom Software Development for Real-Estate, Hosting providers, Workflow and Business Management Systems.

Also see: TransparentProxy

Also see: Resizing a Form has always been a pain in the rectum…

Also, we knew we eventually needed to support hosts which manage all the memory in the process and which use non-preemptive scheduling of tasks and perhaps even light-weight fibers rather than OS threads. So we added some rudimentary (and alas inadequate) APIs for fibers and memory control. This invariably happens when you add features that you think you will eventually need, rather than features that someone is actually using and giving feedback on.

If you look closely at the V1 and V1.1 hosting APIs, you really see what we needed to support ASP.NET and a few other scenarios, like ones involving EnterpriseServices, Internet Explorer or VSA, plus some rudimentary guesses at what we might need to coexist properly inside SQL Server.

Obviously in Whidbey we have refined those guesses about SQL Server into hard requirements. And we tried very hard to generalize each extension that we added for SQL Server, so that it would be applicable to many other hosting scenarios. In fact, it’s amazing that the SQL Server team still talks to us – whenever they ask for anything, we always say No and give them something that works a lot better for other hosts and not nearly so well for SQL Server.

Live Person Software: Turn website visitors into your customers.

Also see: TransparentProxy

Also see: Compatability

Also see: Music and Movies - Give Away the Soundtrack

Also see: Exception Handling in Running a Business

Also see: Spring Web Flow features and feedback request

Also see: TransparentProxy

Also see: Spring Web Flow features and feedback request

Also see: Big in Japan

Also see: Resizing a Form has always been a pain in the rectum…

In our next release (Whidbey), we’ve made a real effort to clean up the existing hosting support and to dramatically extend it for a number of new scenarios. Therefore I’m not going to spend any more time discussing those original V1 & V1.1 hosting APIs, except to the extent that they are still relevant to the following Whidbey hosting discussion.

Also I’m going to skip over all the general introductory topics like “When to host” since they were the source of my writer’s block. Instead, I’m going to leap into some of the more technically interesting topics. Maybe after we’ve studied various details we can step back and see some general guidelines.

Threading and Synchronization

One of the most interesting challenges we struggled with during Whidbey was the need to cooperate with SQL Server’s task scheduling. SQL Server can operate in either thread mode or fiber mode. Most customers run in thread mode, but SQL Server can deliver its best numbers on machines with lots of CPUs when it’s running in fiber mode. That gap between thread and fiber mode has been closing as the OS addresses issues with its own preemptive scheduler.

Custom Software Solutions. Billing and Invoicing Solutions, eCommerce and Website design.

Also see: LoadFrom’s Second Bind

Also see: Spring Web Flow features and feedback request

A few years ago, I ran some experiments to see how many threads I could create in a single process. Not surprisingly, after almost 2000 threads I ran out of address space in the process. That’s because the default stack size on NT is 1 MB and the default user address space is 2 GB. (Starting with V1.1, the CLR can load into LARGEADDRESSAWARE processes and use up to 3 GB of address space). If you shrink the default stack size, you can create more than 2000 threads before hitting the address space limit. I see stack sizes of 256 KB in the SQL Server process on my machine, clearly to reduce this impact on process address space.

Of course, address space isn’t the only limit you can hit. Even on the 4 CPU server box I was experimenting with, the real memory on the system was inadequate for the working set being used. With enough threads, I exceeded real memory and experienced paging. (Okay, it was actually thrashing). But nowadays there are plenty of servers with several GB of real – and real cheap – memory, so this doesn’t have to be an issue.

Multisoft Group: Custom software solutions for your business.

Also see: Fix ReturnUrl When Sharing Forms Authentication with Multiple Web Applications

In my experiments, I simulated server request processing using an artificial work load that combined blocking, allocation, CPU-intensive computation, and a reasonable memory reference set using a mixture of both shared and per-request allocations. In the first experiments, all the threads were ready to run and all of them had equal priority. The result of this was that all threads were scheduled in a round-robin fashion on those 4 CPUs. Since the Windows OS schedules threads preemptively, each thread would execute until it either needed to block or it exceeded its quantum. With hundreds or even thousands of threads, each context switch was extremely painful. That’s because most of the memory used by that thread was so cold in the cache, having been fully displaced by the hundreds of threads that ran before it.

As we all know, modern CPUs are getting faster and faster at raw computation. And they have more and more memory available to them. But access to that memory is getting relatively slower each year. By that, I mean that a single memory access costs the equivalent of an increasing number of instructions. One of the ways the industry tries to mitigate that relative slowdown is through a cache hierarchy. Modern X86 machines have L1, L2 and L3 levels of cache, ordered from fastest and smallest to slowest and largest.

Softwre Development for small and middle size companies. World-class software applications.

(Other ways we try to mitigate the slowdown is by increasing the locality of our data structures and by pre-fetching. If you are a developer, hopefully you already know about locality. In the unmanaged world, locality is entirely your responsibility. In the managed world, you get some locality benefits from our environment – notably the garbage collector, but also the auto-layout of the class loader. Yet even in managed code, locality remains a major responsibility of each developer).

Unfortunately, context switching between such a high number of threads will largely invalidate all those caches. So I changed my simulated server to be smarter about dispatching requests. Instead of allowing 1000 requests to execute concurrently, I would block 996 of those requests and allow 4 of them to run. This makes life pretty easy for the OS scheduler! There are four CPUs and four runnable threads. It’s pretty obvious which threads should run.

Developing Customer Relationship Management Solutions. Web, e-Commerce, Database Design and Software Development.

Also see: Note to self: Blog about using Service Broker

Also see: ASP.NET MVC in CodePlex and Extensible Unit Testing

Also see: Quaker votes

Also see: Compatability

Not only will the OS keep those same four threads executing, it will likely keep them affinitized to the same CPUs. When a thread moves from one CPU to another, the new CPU needs to fill all the levels of cache with data appropriate to the new thread. However, if we can remain affinitized, we can enjoy all the benefits of a warm cache. The OS scheduler attempts to run threads on the CPU that last ran them (soft affinity). But in practice this soft affinity is too soft. Threads tend to migrate between CPUs far more than we would like. When the OS only has 4 runnable threads for its 4 CPUs, the amount of migration seemed to drop dramatically.

Incidentally, Windows also supports hard affinity. If a thread is hard affinitized to a CPU, it either runs on that CPU or it doesn’t run. The CLR can take advantage of this when the GC is executing in its server mode. But you have to be careful not to abuse hard affinity. You certainly don’t want to end up in a situation where all the “ready to run” threads are affinitized to one CPU and all the other CPUs are necessarily stalled.

Help Desk Software: Next generation of Live Chat. Jabber/XMPP Live Chat Service for your website.

Also see: Memory Model

Also see: Mix 08 Sessions Published

Also, it’s worth mentioning the impact of hyper-threading or NUMA on affinity. On traditional SMP, our choices were pretty simple. Either our thread ran on its ideal processor, where we are most likely to see all the benefits of a warm cache, or it ran on some other processor. All those other processor choices can be treated as equally bad for performance. But with hyper-threading or NUMA, some of those other CPUs might be better choices than others. In the case of hyper-threading, some logical CPUs are combined into a single physical CPU and so they share access to the same cache memory at some level in the cache hierarchy. For NUMA, the CPUs may be arranged in partitions (e.g. hemispheres on some machines), where each partition has faster access to some memory addresses and slower access to other addresses. In all these cases, there’s some kind of gradient from the very best CPU(s) for a thread to execute on, down to the very worst CPU(s) for that particular thread. The world just keeps getting more interesting.

Custom Software Development for Real-Estate, Hosting providers, Workflow and Business Management Systems.

Also see: Exception Handling in Running a Business

Anyway, remember that my simulated server combined blocking with other operations. In a real server, that blocking could be due to a web page making a remote call to get rows from a database, or perhaps it could be blocking due to a web service request. If my server request dispatcher only allows 4 requests to be in flight at any time, such blocking will be a scalability killer. I would stall a CPU until my blocked thread is signaled. This would be intolerable.

Many servers address this issue by releasing some multiple of the ideal number of requests simultaneously. If I have 4 CPUs dedicated to my server process, then 4 requests is the ideal number of concurrent requests. If there’s “moderate” blocking during the processing of a typical request, I might find that 8 concurrent requests and 8 threads is a good tradeoff between more context switching and not stalling any CPUs. If I pick too high of a multiple over the number of CPUs, then context switching and cache effects will hurt my performance. If I pick too low a multiple, then blocking will stall a CPU and hurt my performance.

If you look at the heuristics inside the managed ThreadPool, you’ll find that we are constantly monitoring the CPU utilization. If we notice that some CPU resources are being wasted, we may be starving the system by not doing enough work concurrently. When this is detected, we are likely to release more threads from the ThreadPool in order to increase concurrency and make better use of the CPUs. This is a decent heuristic, but it isn’t perfect. For instance, CPU utilization is “backwards looking.” You actually have to stall a CPU before we will notice that more work should be executed concurrently. And by the time we’ve injected extra threads, the stalling situation may already have passed.

The OS has a better solution to this problem. IO Completion Ports have a direct link to the blocking primitives in Win32. When a thread is processing a work item from a completion port, if that thread blocks efficiently through the OS, then the blocking primitive will notify the completion port that it should release another thread. (Busy waiting instead of efficient blocking can therefore have a substantial impact on the amount of concurrency in the process). This feedback mechanism with IO Completion Ports is far more immediate and effective than the CLR’s heuristic based on CPU utilization. But in fairness I should point out that if a managed thread performs managed blocking via any of the managed blocking primitives (contentious Monitor.Enter, WaitHandle.WaitOne/Any/All, Thread.Join, GC.WaitForPendingFinalizers, etc.), then we have a similar feedback mechanism. We just don’t have hooks into the OS, so we cannot track all the blocking operations that occur in unmanaged code.

Of course, in my simulated server I didn’t have to worry about “details” like how to track all OS blocking primitives. Instead, I postulated a closed world where all blocking had to go through APIs exposed by my server. This gave me accurate and immediate information about threads either beginning to block or waking up from a blocking operation. Given this information, I was able to tweak my request dispatcher so it avoided any stalling by injecting new requests as necessary.

Although it’s possible to completely prevent stalling in this manner, it’s not possible to prevent context switches. Consider what happens on a 1 CPU machine. We release exactly one request which executes on one thread. When that thread is about to block, we release a second thread. So far, it’s perfect. But when the first thread resumes from its blocking operation, we now have two threads executing concurrently. Our request dispatcher can “retire” one of those threads as soon as it’s finished its work. But until then we have two threads executing on a single CPU and this will impact performance.

I suppose we could try to get ruthless in this situation, perhaps by suspending one of the threads or reducing its priority. In practice, it’s never a good idea to suspend an executing thread. If that thread holds any locks that are required by other concurrent execution, we may have triggered a deadlock. Reducing the priority might help and I suspect I played around with that technique. To be honest, I can’t remember that far back.

We’ll see that SQL Server can even solve this context switching problem.

Oh yeah, SQL Server

So what does any of this have to do with SQL Server?

Not surprisingly, the folks who built SQL Server know infinitely more than me about how to get the best performance out of a server. And when the CLR is inside SQL Server, it must conform to their efficient design. Let’s look at their thread mode, first. Fiber mode is really just a refinement over this.

Incoming requests are carried on threads. SQL Server handles a lot of simultaneous requests, so there are a lot of threads in the process. With normal OS “free for all” scheduling, this would result in way too many context switches, as we have seen. So instead those threads are affinitized to a host scheduler / CPU combination. The scheduler tries to ensure that there is one unblocked thread available at any time. All the other threads are ideally blocked. This gives us the nirvana of 100% busy CPUs and minimal context switches. To achieve this nirvana, all the blocking primitives need to cooperate with the schedulers. Even if an event has been signaled and a thread is considered by the application to be “ready to run”, the scheduler may not choose to release it, if the scheduler’s corresponding CPU is already executing another thread. In this manner, the blocking primitive and the scheduler are tightly integrated.

When I built my simulated server, I was able to achieve an ideal “closed world” where all the synchronization primitives were controlled by me. SQL Server attempts the same thing. If a thread needs to block waiting for a data page to be read, or for a page or row latch to be released, that blocking occurs through the SQL Server scheduler. This guarantees that exactly one thread is available to run on each CPU, as we’ve seen.

Of course, execution of managed code also hits various blocking points. Monitor.Enter (‘lock’ in C# and ‘SyncLock’ in VB.NET) is a typical case. Other cases include waiting for a GC to complete, waiting for class construction or assembly loading or type loading to occur, waiting for a method to be JITted, or waiting for a remote call or web service to return. For SQL Server to hit their performance goals and to avoid deadlocks, the CLR must route all of these blocking primitives to SQL Server (or any other similar host) through the new Whidbey hosting APIs.

Leaving the Closed World

But what about synchronization primitives that are used for coordination with unmanaged code and which have precise semantics that SQL Server cannot hope to duplicate? For example, WaitHandle and its subtypes (like Mutex, AutoResetEvent and ManualResetEvent) are thin wrappers over the various OS waitable handles. These primitives provide atomicity guarantees when you perform a WaitAll operation on them. They have special behavior related to message pumping. And they can be used to coordinate activity across multiple processes, in the case of named primitives. It’s unrealistic to route operations on WaitHandle through the hosting APIs to some equivalent host-provided replacements.

This issue with WaitHandle is part of a more general problem. What happens if I PInvoke from managed code to an OS service like CoInitialize or LoadLibrary or CryptEncrypt? Do those OS services block? Well, I know that LoadLibrary will have to take the OS loader lock somewhere. I could imagine that CoInitialize might need to synchronize something, but I have no real idea. One thing I am sure of: if any blocking happens, it isn’t going to go through SQL Server’s blocking primitives and coordinate with their host scheduler. The idealized closed world that SQL Server needs has just been lost.

The solution here is to alert the host whenever a thread “leaves the runtime”. In other words, if we are PInvoking out, or making a COM call, or the thread is otherwise transitioning out to some unknown unmanaged execution, we tell the host that this is happening. If the host is tracking threads as closely as SQL Server does, it can use this event to disassociate the thread from the host scheduler and release a new thread. This ensures that the CPU stays busy. That’s because even if the disassociated thread blocks, we’ve released another thread. This newly released thread is still inside our closed world, so it will notify before it blocks so we can guarantee that the CPU won’t stall.

Wait a second. The CLR did a ton of work to re-route most of its blocking operations through the host. But we could have saved almost that entire ton of engineering effort if we had just detached the thread from the host whenever SQL Server called into managed code. That way, we could freely block and we wouldn’t disrupt the host’s scheduling decisions.

This is true, but it won’t perform as well as the alternative. Whenever a thread disassociates from a host scheduler, another thread must be released. This guarantees that the CPU is busy, but it has sacrificed our nirvana of only having a single runnable thread per CPU. Now we’ve got two runnable threads for this CPU and the OS will be preemptively context-switching between them as they run out of quantum.

If a significant amount of the processing inside a host is performed through managed code, this would have a serious impact on performance.

Indeed, if a significant amount of the processing inside a host is performed in unmanaged code, called via PInvokes or COM calls or other mechanisms that “leave the runtime”, this too can have a serious impact on performance. But, for practical purposes, we expect most execution to remain inside the host or inside managed code. The amount of processing that happens in arbitrary unmanaged code should be low, especially over time as our managed platform grows to fill in some of the current gaps.

Of course, some PInvokes or COM calls might be to services that were exported from the host. We certainly don’t want to disassociate from the host scheduler every time the in-process ADO provider performs a PInvoke back to SQL Server to get some data. This would be unnecessary and expensive. So there’s a way for the host to control which PInvoke targets perform a “leave runtime” / “return to runtime” pair and which ones are considered to remain within the closed world of our integrated host + runtime.

Even if we were willing to tolerate the substantial performance impact of considering all of the CLR to be outside the host’s closed world (i.e. we disassociated from the host’s scheduler whenever we ran managed code), this approach would be inadequate when running in fiber mode. That’s because of the nasty effects which thread affinity can have on a fiber-based system.

Fiber Mode

As we’ve seen, SQL Server and other “extreme” hosts can ensure that at any time each CPU has only a single thread within the closed world that is ready to run. But when SQL Server is in thread mode, there are still a large number of threads that aren’t ready to run. It turns out that all those blocked threads impose a modest cost upon the OS preemptive scheduler. And that cost becomes an increasing consideration as the number of CPUs increases. For 1, 2, 4 and probably 8 CPU machines, fiber mode isn’t worth the headaches we’re about to discuss. But by the time you get to a larger machine, you might achieve something like a 20% throughput boost by switching to fiber mode. (I haven’t seen real numbers in a year or two, so please take that 20% as a vague ballpark).

Fiber mode simply eliminates all those extra threads from any consideration by the OS. If you stay within the idealized nirvana (i.e. you don’t perform a “leave runtime” operation), there is only one thread for each host scheduler / CPU. Of course, there are many stacks / register contexts and each such stack / register context corresponds to an in-flight request. When a stack is ready to run, the single thread switches away from whatever stack it was running and switches to the new stack. But from the perspective of the OS scheduler, it just keeps running the only thread it knows about.

So in both thread mode and fiber mode, SQL Server uses non-preemptive host scheduling of these tasks. This scheduling happens in user mode, which is a distinct advantage over the OS preemptive scheduling which happens in kernel mode. The only difference is whether the OS scheduler is aware of all the tasks on the host scheduler, or whether they all look like a single combined thread – albeit with different stacks and register contexts.

But the impact of this difference is significant. First, it means that there is an M:N relationship between stacks (logical CLR threads) and OS threads. This is M:N because multiple stacks will execute on a single thread, and because the specially nominated thread that carries those stacks can change over time. This change in the nominated thread occurs as a consequence of those “leave runtime” calls. Remember that when a thread leaves the runtime, we inform the host which disassociates the thread from the host scheduler. A new thread is then created or obtained from a short list of already-created threads. This new thread then picks up the next stack that is ready to run. The effect is that this stack has migrated from the original disassociated thread to the newly nominated thread.

This M:N relationship between stacks and OS threads causes problems everywhere that thread affinity would normally occur. I’ve already mentioned CPU affinity when discussing how threads are associated with CPUs. But now I’m talking about a different kind of affinity. Thread affinity is the association between various programmatic operations and the thread that these operations must run on. For example, if you take an OS critical section by calling EnterCriticalSection, the resulting ownership is tied to your thread. Sometimes developers say that the OS critical section is scoped to your thread. You must call LeaveCriticalSection from that same thread.

None of this is going to work properly if your logical thread is asynchronously and apparently randomly migrating between different physical threads. You’ll successfully take the critical section on one logical thread. If you attempt to recursively acquire this critical section, you will deadlock if a migration has intervened. That’s because it will look like a different physical thread is actually the owner.

Imagine writing some hypothetical code inside the CLR:

EnterCriticalSection(pCS);

If (pGlobalBlock == NULL)

pGlobalBlock = Alloc(count);

LeaveCriticalSection(pCS);

Obviously any real CLR code would be full of error handling, including a ‘finally’ clause to release the lock. And we don’t use OS critical sections directly since we typically reflect them to an interested host as we’ve discussed. And we instrument a lot of this stuff, including spinning during lock acquisition. And we wrap the locks with lots of logic to avoid deadlocks, including GC-induced deadlocks. But let’s ignore all of the goop that would be necessary for real CLR code.

It turns out that the above code has a thread affinity problem. Even though SQL Server’s fiber scheduling is non-preemptive, scheduling decisions can still occur whenever we call into the host. For reasons that I’ll explain later, all memory allocations in the CLR have the potential to call into the host and result in scheduling. Obviously most allocations will be satisfied locally in the CLR without escalation to the host. And most escalations to the host still won’t cause a scheduling decision to occur. But from a correctness perspective, all allocations have the potential to cause scheduling.

Other places where thread affinity can bite us include:

The OS Mutex and the managed System.Threading.Mutex wrapper.

LoadLibrary and DllMain interactions. As I’ve explained in my blog entry on Shutdown, DllMain notifications occur on a thread which holds the OS loader lock.

TLS (thread local storage). It’s worth mentioning that, starting with Windows Server 2003, there are new FLS (fiber local storage) APIs. These APIs allow you to associate state with the logical rather than the physical thread. When a fiber is associated with a thread for execution (SwitchToFiber), the FLS is automatically moved from the fiber onto the thread. For managed TLS, we now move this automatically. But we cannot do this unconditionally for all the unmanaged TLS.

Thread culture or locale, the impersonation context or user identity, the COM+ transaction context, etc. In some sense, these are just special cases of thread local storage. However, for historical reasons it isn’t possible to solve these problems by moving them to FLS.

Taking control of a thread for GC, Abort, etc. via the OS SuspendThread() service.

Any use of ThreadId or Thread Handle. This includes all debugging.

“Hand-rolled” locks that we cannot discover or reason about, and which you have inadvertently based on the physical OS thread rather than the logical thread or fiber.

Various PInvokes or COM calls that might end up in unmanaged code with affinity requirements. For instance, MSHTML can only be called on STA threads which are necessarily affinitized. Of course, there is no list of all the APIs that have odd threading behavior. It’s a minefield out there.

Solving affinity issues is relatively simple. The hard part is identifying all the places. Note that the last two bullet items are actually the application’s responsibility to identify. Some application code might appear to execute correctly when logical threads and OS threads are 1:1. But when a host creates an M:N relationship, any latent application bugs will be exposed.

In many cases, the easiest solution to a thread affinity issue is to disassociate the thread from the host’s scheduler until the affinity is no longer required. The hosting APIs provide for this, and we’ve taken care of it for you in many places – like System.Threading.Mutex.

Before we finish our discussion of locking, there’s one more aspect worth mentioning. In an earlier blog, I have mentioned the limited deadlock detection and deadlock breaking which the CLR performs when executing class constructors or JITting.

Except for this limited case, the CLR doesn’t concern itself with application-level deadlocks. If you write some managed code that takes a set of locks in random order, resulting in a potential deadlock, we consider that to be your application bug. But some hosts may be more helpful. Indeed, SQL Server has traditionally detected deadlocks in all data accesses. When a deadlock occurs, SQL Server selects a victim and aborts the corresponding transaction. This allows the other requests implicated in the deadlock to proceed.

With the new Whidbey hosting APIs, it’s possible for the host to walk all contentious managed locks and obtain a graph of the participants. This support extends to locking through our Monitor and our ReaderWriterLock. Clearly, an application could perform locking through other means. For example, an AutoResetEvent can be used to simulate mutual exclusion. But it’s not possible for such locks to be included in the deadlock algorithms, since there isn’t a strong notion of lock ownership that we can use.

Once the host has selected a deadlock victim, it must cause that victim to abort its forward progress somehow. If the victim is executing managed code, some obvious ways to do this include failing the lock attempt (since the thread is necessarily blocking), aborting the thread, or even unloading the AppDomain. We’ll return to the implications of this choice in the section on Reliability below.

Finally, it’s interesting to consider how one might get even better performance than what SQL Server has achieved. We’ve seen how fiber mode eliminates all the extra threads, by multiplexing a number of stacks / register contexts onto a single thread. What happens if we then eliminate all those fibers? For a dedicated server, we can achieve even better performance by forcing all application code to maintain its state outside of a thread’s stack. This allows us to use a single thread per CPU which executes user requests by processing them on its single dedicated stack. All synchronous blocking is eliminated by relying on asynchronous operations. The thread never yields while holding its stack pinned. The amount of memory required to hold an in-flight request will be far less than a 256 KB stack reservation. And the cost of processing an asynchronous completion through polling can presumably be less than the cost of a fiber context switch.

If all you care about is performance, this is an excellent way to build a server. But if you need to accommodate 3^rd party applications inside the server, this approach is questionable. Most developers have a difficult time breaking their logic into segments which can be separately scheduled with no stack dependencies. It’s a tedious programming model. Also, the underlying Windows platform still contains a lot of blocking operations that don’t have asynchronous variants available. WMI is one example.

Memory Management

Servers must not page.

Like all rules, this one isn’t strictly true. It is actually okay to page briefly now and then, when the work load transitions from one steady state to another. But if you have a server that is routinely paging, then you have driven that server beyond its capacity. You need to reduce the load on the server or increase the server’s memory capacity.

At the same time, it’s important to make effective use of the memory capacity of a server. Ideally, a database would store the entire database contents in memory. This would allow it to avoid touching the disk, except to write the durable log that protects it from data loss and inconsistency in the face of catastrophic failure. Of course, the 2 or 3 GB limit of Win32 is far too restrictive for most interesting databases. (SQL Server can use AWE to escape this limit, at some cost). And even the address limits of Win64 are likely to be exceeded by databases presently. That’s because Win64 does not give you a full 64 bits of addressing and databases are already heading into the petabytes.

So a database needs to consider all the competing demands for memory and make wise decisions about which ones to satisfy. Historically, those demands have included the buffer cache which contains data pages, compiled query plans, and all those thread stacks. When the CLR is loaded into the process, significant additional memory is required for the GC heap, application code, and the CLR itself. I’m not sure what techniques SQL Server uses to trade off the competing demands for memory. Some servers carve memory up based on fixed ratios for the different broad uses, and then rely on LRU within each memory chunk. Other servers assign a cost to each memory type, which indicates how expensive it would be to regenerate that memory. For example, in the case of a data page, that cost is an IO.

Some servers use elaborate throttling of inbound requests, to keep the memory load reasonable. This is relatively easy to do when all requests are comparable in terms of their memory and CPU requirements. But if some queries access a single database page and other queries touch millions of rows, it would be hard to factor this into a throttling decision that is so far upstream from the query processor. Instead, SQL Server tends to accept a large number of incoming requests and process them “concurrently.” We’ve already seen in great detail why this concurrent execution doesn’t actually result in preemptive context switching between all the corresponding tasks. But it is still the case that each request will hold onto some reference set of memory, even when the host’s non-preemptive scheduler has that request blocked.

If enough requests are blocked while holding onto significant unshared memory, then the server process may find itself over-committed on memory. At this point, it could page – which hurts performance. Or it could kill some of the requests and free up the resources they are holding onto. This is an unfortunate situation, because we’ve presumably already devoted resources like the CPU to get the request to its current state of partial completion. If we throw away the request, all that work was wasted. And the client is likely to resubmit the request, so we will have to repeat all that work soon.

Nevertheless, if the server is over-committed and it’s not practical to recover more memory by e.g. shrinking the number of pages devoted to the buffer cache, then killing in-flight requests is a sound strategy. This is particularly reasonable in database scenarios, since the transactional nature of database operations means that we can kill requests at any time and with impunity.

Unfortunately, the world of arbitrary managed execution has no transactional foundation we can rely on. We’ll pick up this issue again below, in the section on Reliability.

It should be obvious that, if SQL Server or any other host is going to make wise decisions about memory consumption on a “whole process” basis, that host needs to know exactly how much memory is being used and for what purposes. For example, before the host unloads an AppDomain as a way of backing out of an over-committed situation, the host needs some idea of how many megabytes this unload operation is likely to deliver.

In the reverse direction, the host needs to be able to masquerade as the operating system. For instance, the CLR’s GC monitors system memory load and uses this information in its heuristics for deciding when to schedule a collection. The host needs a way to influence these collection decisions.

SQL Server and ASP.NET

Clearly a lot of work went into threading, synchronization and memory management in SQL Server. One obvious question to ask is how ASP.NET compares. They are both server products from Microsoft and they both execute managed code. Why didn’t we need to add all this support to the hosting interfaces in V1 of the CLR, so we could support ASP.NET?

I think it’s fair to say that ASP.NET took a much simpler approach to the problem of building a scalable server. To achieve efficient threading, they rely on the managed ThreadPool’s heuristics to keep the CPUs busy without driving up too many context switches. And since the bulk of memory allocations are due to the application, rather than the ASP.NET infrastructure (in other words, they aren’t managing large shared buffer pools for data pages), it’s not really possible for ASP.NET to act as a broker for all the different memory consumers. Instead, they just monitor the total memory load, and recycle the worker process if a threshold is exceeded.

(Incidentally, V1 of ASP.NET and the CLR had an unfortunate bug with the selection of this threshold. The default point at which ASP.NET would recycle the process was actually a lower memory load than the point at which the CLR’s GC would switch to a more aggressive schedule of collections. So we were actually killing the worker process before the CLR had a chance to deliver more memory back to the application. Presumably in Whidbey this selection of default thresholds is now coordinated between the two systems.)

How can ASP.NET get away with this simpler approach?

It really comes down to their fundamental goals. ASP.NET can scale out, rather than having to scale up. If you have more incoming web traffic, you can generally throw more web servers at the problem and load balance between them. Whereas SQL Server can only scale out if the data supports this. In some cases, it does. There may be a natural partitioning of the data, like access to the HotMail mailbox for a particular incoming user. But in too many other cases, the data cannot be sufficiently partitioned and the server must be scaled up. On X86 Windows, the practical limit is a 32-way CPU with a hard limit of 3 GB of user address space. If you want to keep increasing your work load on a single box, you need to use every imaginative trick – like fibers or AWE – to eke out all possible performance.

There’s also an availability issue. ASP.NET can recycle worker processes quite quickly. And if they have scaled out, recycling a worker process on one of the computers in the set will have no visible effect on the availability of the set of servers. But SQL Server may be limited to a single precious process. If that process must be recycled, the server is unavailable. And recycling a database is more expensive than recycling a stateless ASP.NET worker process, because transaction logs must be replayed to move the database forwards or backwards to a consistent state.

The short answer is, ASP.NET didn’t have to do all the high tech fancy performance work. Whereas SQL Server was forced down this path by the nature of the product they must build.

Reliability

Well, if you haven’t read my earlier blogs on asynchronous exceptions, or if – like me – you read the Reliability blog back in June and don’t remember what it said – you might want to review it quickly at http://blogs.msdn.com/cbrumme/archive/2003/06/23/51482.aspx.

The good news is that we’ve revisited the rules for ThreadAbortException in Whidbey, so that there is now a way to abort a thread without disturbing any backout code that it is currently running. But it’s still the case that asynchronous exceptions can intrude at fairly arbitrary spots in the execution.

Anyway, the availability goals of SQL Server place some rather difficult requirements on the CLR. Sure, we were pretty solid in V1 and V1.1. We ran a ton of stress and – if you avoided stack overflow, running out of memory, and any asynchronous exceptions like Thread.Abort – we could run applications indefinitely. We really were very clean.

One problem with this is that “indefinitely” isn’t long enough for SQL Server. They have a noble goal of chasing 5 9’s and you can’t get there with loose statements like “indefinitely”. Another problem is that we can no longer exclude OutOfMemoryException and ThreadAbortException from our reliability profile. We’ve already seen that SQL Server tries to use 100% of memory, without quite triggering paging. The effect is that SQL Server is always on the brink of being out of memory, so allocation requests are frequently being denied. Along the same lines, if the server is loaded it will allow itself to become over-committed on all resources. One strategy for backing out of an over-commitment is to abort a thread (i.e. kill a transaction) or possibly unload one or more AppDomains.

Despite this stressful abuse, at no time can the process terminate.

The first step to achieve this was to harden the CLR so that it was resilient to any resource failures. Fortunately we have some extremely strong testers. One tester built a system to inject a resource failure in every allocator, for every unique logical call stack. This tests every distinct backout path in the product. This technique can be used for unmanaged and managed (FX) code. That same tester is also chasing any unmanaged leaks by applying the principles of a tracing garbage collector to our unmanaged CLR data structures. This technique has already exposed a small memory leak that we shipped in V1 of the CLR – for the “Hello World” application!

With testers like that, you better have a strong development team too. At this point, I think we’ve annotated the vast majority of our unmanaged CLR methods with reliability contracts. These are a bit like Eiffel pre- and post-conditions and they provide machine-verifiable statements about each method’s behavior with respect to GC, exceptions, and other fundamental operations. These contracts can be used during test coverage (and, in some cases, during static scans of the binary images) to test for conformance.

The bottom line is that the next release of CLR should be substantially more robust in the face of resource errors. Leaving aside stack overflows and focusing entirely on the unmanaged runtime, we are shooting for perfection. Even for stack overflow, we expect to get very, very close. And we have the mechanisms in place that allow us to be rigorous in chasing after these goals.

But what about all of the managed code?

Will FX be as robust as the unmanaged CLR? And how can we possibly hold 3^rd party authors of stored procedures or user defined functions to that same high bar? We want to enable a broad class of developers to write this sort of code, and we cannot expect them to perform many hundreds of hours of stress testing and fault injection on each new stored procedure. If we’re chasing 5 9’s by requiring every external developer to write perfect code, we should just give up now.

Instead, SQL Server relies on something other than perfect code. Consider how SQL Server worked before it started hosting the CLR:

The vast majority of execution inside SQL Server was via Transact SQL or TSQL. Any application written in TSQL is inherently scalable, fiber-aware, and robust in the face of resource errors. Any computation in TSQL can be terminated with a clean transaction abort.

Unfortunately, TSQL isn’t expressive enough to satisfy all application needs. So the remaining applications were written in extended stored procedures or xprocs. These are typically unmanaged C++. Their authors must be extremely sophisticated, because they are responsible for integrating their execution with the unusual threading environment and resource rules that exist inside SQL Server. Throw in the rules for data access and security (which I won’t be discussing in this blog) and it takes superhuman knowledge and skill to develop a bug-free xproc.

In other words, you had a choice of well-behaved execution and limited expression (TSQL), or the choice of arbitrary execution coupled with a very low likelihood that you would get it right (xprocs).

One of the shared goals of the SQL Server and CLR teams in Whidbey was to eliminate the need for xprocs. We wanted to provide a spectrum of choices to managed applications. In Whidbey, that spectrum consists of three buckets for managed code:

Safe

Code in this bucket is the most constrained. In fact, the host constrains it beyond what the CLR would normally allow to code that’s only granted SecurityPermissionFlag.Execution. So this code must be verifiably typesafe and has a reduced grant set. But it is further constrained from defining mutable static fields, from creating or controlling threads, from using the threadpool, etc. The goal here is to guide the code to best practices for scalability and robustness within the SQL Server or similar hosted environments. In the case of SQL Server, this means that all state should be stored in the database and that concurrency is controlled through transactions against the data. However, it’s important to realize that these additional constraints are not part of the Security system and they may well be subvertible. The constraints are simply speedbumps (not roadblocks) which guide the application code away from potentially non-scalable coding techniques and which encourage best practices.

External Access

Code in this bucket should be sufficient for replacing most xprocs. Such code must also be verifiably typesafe, but it is granted some additional permissions. The exact set of permissions is presumably subject to change until Yukon ships, but it’s likely to allow access to the registry, the file system, and the network.

Unsafe

This is the final managed escape hatch for writing code inside SQL Server. This code does not have to be verifiable. It has FullTrust (with the possible exception of UIPermission, which makes no sense within the database). This means that it can do anything the most arbitrary xproc can do. However, it is much more likely to work properly, compared to that xproc. First, it sits on top of a framework that has been designed to work inside the database. Second, the code has all the usual benefits of managed code, like a memory manager that’s based on accurate reachability rather than on programmer correctness. Finally, it is executing on a runtime that understands the host’s special rules for resource management, synchronization, threading, security, etc.

For code in the Safe bucket, you may be wondering how a host could constrain code beyond SecurityPermissionFlag.Execution. There are two techniques available for this:

1) Any assembly in the ‘Safe’ subset could be scanned by a host-provided pre-verifier, to check for any questionable programming constructs like the definition of mutable static fields, or the use of reflection. This raises the obvious question of how the host can interject itself into the binding process and guarantee that only pre-verified assemblies are loaded. The new Whidbey hosting APIs contain a Fusion loader hook mechanism, which allows the host to abstract the notion of an assembly store, without disturbing all our normal loader policy. You can think of this as the natural evolution of the AppDomain.AssemblyResolve event. SQL Server can use this mechanism to place all application assemblies into the database and then deliver them to the loader on demand. In addition to enabling pre-verification, the loader hooks can also be used to ensure that applications inside the database are not inadvertently broken or influenced by changes outside the database (e.g. changes to the GAC). In fact, you could even copy a database from one machine to another and theoretically this could automatically transfer all the assemblies required by that database.

2) The Whidbey hosting APIs provide controls over a new Host Protection Attribute (HPA) feature. Throughout our frameworks, we’ve decorated various unprotected APIs with an appropriate HPA. These HPAs indicate that the decorated API performs a sensitive operation like Synchronization or Thread Control. For instance, use of the ThreadPool isn’t considered a secure operation. (At some level, it is a risk for Denial of Service attacks, but DOS remains an open design topic for our managed platform). If code is running outside of a host that enables these HPAs, they have no effect. Partially trusted code, including code that only has Execution permission, can still call all these APIs. But if a host does enable these attributes, then code with insufficient trust can no longer call these APIs directly. Indirect calls are still permitted, and in this sense the HPA mechanism is similar to the mechanism for LinkDemands.

Although HPAs use a mechanism that is similar to LinkDemands, it’s very important to distinguish the HPA feature – which is all about programming model guidance – from any Security feature. A great way to illustrate this distinction is Monitor.Enter.

Ignoring HPAs, any code can call Monitor.Enter and use this API to synchronize with other threads. Naturally, SQL Server would prefer that most developers targeting their environment (including all the naïve ones) should rely on database locks under transaction control for this sort of thing. Therefore they activate the HPA on this class:

[HostProtection(Synchronization=true, ExternalThreading=true)]

public sealed class Monitor

{

…

[MethodImplAttribute(MethodImplOptions.InternalCall)]

public static extern void Enter(Object obj);

However, devious code in the ‘Safe’ bucket could use a HashTable as an alternate technique for locking. If you create a synchronized HashTable and then perform inserts or lookups, your Object.Equals and GetHashCode methods will be called within the lock that synchronizes the HashTable. The BCL developers were smart enough to realize this, and they added another HPA:

public class Hashtable : IDictionary, ISerializable,

IDeserializationCallback, ICloneable

{

…

[HostProtection(Synchronization=true)]

public static Hashtable Synchronized(Hashtable table) {

if (table==null)

throw new ArgumentNullException(”table”);

return new SyncHashtable(table);

}

Are there other places inside the frameworks where it’s possible to trick an API into providing synchronization for its caller? Undoubtedly there are, but we aren’t going to perform exhaustive audits of our entire codebase to discover them all. As we find additional APIs, we will decorate them with HPAs, but we make no guarantees here.

This would be an intolerable situation for a Security feature, but it’s perfectly acceptable when we’re just trying to increase the scalability and reliability of naively written database applications.

Escalation Policy

I chose the HPA on System.Threading.Monitor for a reason, in the above example. If you’ve read my earlier blogs on Thread.Abort, you know that it’s dangerous to asynchronously abort another thread. That thread could be executing a class constructor, in which case that class is now unavailable throughout the AppDomain. That thread could be in the middle of an update to some shared application state, which would leave the application in an inconsistent state.

In V1 & V1.1, it was not really possible to write code that is robust in the face of asynchronous exceptions like Abort. In Whidbey, we’re now introducing some constructs (Constrained Execution Regions and Critical Finalization) which make it possible to do this. I’m not going to discuss those constructs in this blog. But suffice it to say that, although it makes it possible to write entirely robust code, it doesn’t make it easy. Without a higher level programmatic construct, like transactions, it’s very difficult to write entirely robust code. You must acquire all the resources required for forward progress, tolerating exceptions during this acquisition phase. Then you enter a forward progress phase, which either cannot fail or which unconditionally triggers some compensating backout code upon failure. If compensation is triggered, it must guarantee that the system is returned to a consistent state before it completes.

If you’ve successfully written that sort of code, you know that it’s an onerous discipline. There’s no way that we can expect the greater population of developers to write large bodies of bug-free code based on this plan.

That’s why, in V1 & V1.1, we recommend either using Abort on the current thread (in which case it is not asynchronous) or we recommend using it in conjunction with an AppDomain.Unload (in which case any inconsistent application state is likely to be discarded).

In Whidbey, it is possible to avoid inducing asynchronous Aborts onto threads that are performing backout (i.e. filter, finally, catch or fault blocks) or that hold locks. Our definition of a lock is pretty broad. It includes execution of a class constructor, since all.cctor execution is synchronized according to elaborate rules by the CLR. It also includes Monitor.Enter, Mutex, ReaderWriterLock, etc. Finally, it includes any “hand-rolled” locks that you build, so long as you properly identify them to us.

Our rationale here is that any thread holding a lock may be updating shared state. If a thread isn’t holding a lock, then any update it performs against shared state must be atomic or at least it never leaves that shared state in an inconsistent state. This is strictly a heuristic, but it’s a pretty good one.

If we believe this heuristic, it means that we can use Abort without consequently unloading an AppDomain, if that thread doesn’t hold any locks and isn’t performing any backout. And it just so happens that the bulk of all managed code executing inside SQL Server is in the ‘Safe’ subset – which coincidentally is highly discouraged via HPAs from taking or holding locks.

In other words, code in the ‘Safe’ subset can almost always take an asynchronous exception without affecting any of the execution on other threads in the same AppDomain. This is the case, even though that code was written by developers who don’t understand the deep issues involved with asynchronous exceptions. It further means that if we should catch such a thread at a point where it isn’t safe to inject an asynchronous exception without also unloading the AppDomain, we can identify this window. Once this window is identified, we can either hold off from injecting the exception until this unsafe window has closed, or we can unload the entire AppDomain to eliminate the application inconsistency. The host can decide whether to hold off on the injection or alternatively to proceed with an AppDomain unload, based on criteria like how resource-constrained the host is.

The hosting APIs for making these decisions imperatively would be rather complicated. So the Whidbey hosting APIs provide a declarative mechanism called an escalation policy. This allows the host to express transitions and timeouts that take effect during error conditions. For instance, SQL Server might state that any attempt to Abort a thread should delay if the victim thread holds a lock. But if that delay exceeds 30 seconds, the Abort attempt should be escalated to an AppDomain.Unload. Of course, the feature is more general than SQL Server’s needs. Indeed, the V1 ASP.NET process recycling feature should now be expressible as a particular Whidbey escalation policy.

Winding down

As usual, I didn’t get around to many of the interesting topics. For instance, those guidelines on when and how to host are noticeably absent. And I didn’t explain how to do any simple stuff, like picking concurrent vs. non-concurrent vs. server GC. The above text is completely free of any specific details of what our hosting APIs look like (partly because they are subject to change until Whidbey ships). And I didn’t touch on any hosting topics outside of the hosting APIs, like all of the AppDomain considerations. As you can imagine, there’s also plenty I could have said about Security. For instance, the hosting APIs allow the host to participate in role-based security and impersonation of Windows identities… Oh well.

Fortunately, one of the PMs involved in the Whidbey hosting effort is apparently writing a book on the general topic of hosting. Presumably all these missing topics will be covered there. And hopefully he won’t run into the same issues with writer’s block that I experienced on this topic.

(Indeed, the event that ultimately resolved my writer’s block was that my wife got the flu. When she’s not around, my weekends are boring enough for me to think about work. The reason I’m posting two blogs this weekend is that Kathryn has gone to Maui for the week and has left me behind.)

Finally, the above blog talks about SQL Server a lot.

Hopefully it’s obvious that the CLR wants to be a great execution environment for a broad set of servers. In V1, we focused on ASP.NET. Based on that effort, we automatically worked well in many other servers with no additional work. For example, EnterpriseServices dropped us into their server processes simply by selecting the server mode of our GC. Nothing else was required to get us running efficiently. (Well, we did a ton of other work in the CLR to support EnterpriseServices. But that work was related to the COM+ programming model and infrastructure, rather than their server architecture. We had to do that work whether we ran in their server process or were instead loading EnterpriseServices into the ASP.NET worker process or some other server).

In Whidbey we focused on extending the CLR to meet SQL Server’s needs. But at every opportunity we generalized SQL Server’s requirements and tried to build something that would be more broadly useful. Just as our ASP.NET work enabled a large number of base server hosting scenarios, we hope that our SQL Server work will enable a large number of advanced server hosting scenarios.

If you have a “commercially significant” hosting problem, whether on the server or the client, and you’re struggling with how to incorporate managed code, I would be interested in hearing from you directly. Feel free to drop me an email with the broad outline of what you are trying to achieve, and I’ll try to get you supported. That support might be something as lame as some suggestions from me on how I would tackle the problem. Or at the other extreme, I could imagine more formal support and conceivably some limited feature work. That other extreme really depends on how commercially significant your product is and on how well our business interests align. Obviously decisions like that are far outside my control, but I can at least hook you up with the right people if this seems like a sensible approach.

Okay, one more ‘Finally’. From time to time readers of my blog send me emails asking if there are jobs available on the CLR team. At this moment, we do. Drop me an email if you are interested. It’s an extremely challenging team to work on, but the problems are truly fascinating.

http://blogs.msdn.com/cbrumme/archive/2004/02/21/77595.aspx

When Are Two Algorithms the Same?

Administrator — Mon, 24 Mar 2008 17:00:04 +0000

When Are Two Algorithms the Same? Andreas Blass, Nachum Dershowitz, Yuri Gurevich. February 2008

People usually regard algorithms as more abstract than the programs that implement them. The natural way to formalize this idea is that algorithms are equivalence classes of programs with respect to a suitable equivalence relation. We argue that no such equivalence relation exists.

A bit more philosophical than usual, but the issue is quite relevant to discussions in the field.

It is possible to stipulate any equivalence relation that is considered useful (e.g., equivalence up to local transformations) but the notion of a universally applicable relation is indeed problematic.
http://lambda-the-ultimate.org/node/2729

DevWeek 2008 Cross Platform Silverlight Demos

Administrator — Mon, 24 Mar 2008 12:00:05 +0000

I just finished the Cross Platform.NET on Silverlight talk at DevWeek. Demos can be downloaded from http://www.interact-sw.co.uk/downloads/DevWeek2008XPlatDemos.zip

I’m all done at DevWeek for this year. But if you want to hear more about Silverlight, I’ll be teaching Pluralsight’s Applied Silverlight course in London later this month - running from 31st March. (And the following week I’ll be teaching our Applied WPF course, also in London.)

http://www.interact-sw.co.uk/iangblog/2008/03/12/devweek-xplat-demos

Turning bitboards from potential moves into legal moves, pawn moves, and conditional rules.

Administrator — Mon, 24 Mar 2008 10:48:25 +0000

The BitBoards so far have been astoundingly accurate at producing moves. But even after the moves have been produced they have to be fully validated. Take for instance, a bishop in the middle of the board. The number of potential moves for the bishop is 13 or so, but the number of valid moves, unless no spots are blocked, is much less. Further performing friendly versus non-friendly extension is extremely importan since you can’t move into a friendly position, but you can move into the first occuring non-friendly position (capturing). I’ve found some interesting transformations here, but once I can more fully validate them I’ll start to post their intricacies.

Even more frustrating are the pawns. Pawns are capable of special feats when in their original file (forward by 2), they are allowed capturing moves that are different from their standard movement rules, and they are also allowed the ability to capture en-passant. Deciding where and how to implement these extra conditions is very important. Remember the original blocking square algorithm I implemented for removing invalid moves consisted of:

uint myPieces =…; uint notMine = ~myPieces; uint validMoves = moves & notMine;

This has to be expanded a bit, since notMine actually points to all empty and enemy squares. What we need now is a blocking region for all enemy squares (which we have to store anyway, since eventually the board swaps sides and enemy and friendly are reversed). The valid moves become something like:

uint nonCapture = pawnNonCapture[x]; uint capture = pawnCapture[x];
uint validNonCapture = (~(myPieces & enemyPieces) & nonCapture);
uint validCapture = capture & enemyPieces;
if ( board.enPassant ) { /* special case when prior moves allow */ }
uint validMoves = validNonCapture | validCapture;

Also see: Big in Japan

Also see: TransparentProxy

Also see: Memory Model

Also see: Reporting Services administration changes in Katmai (v.Next)

Also see: Big in Japan

Even with all of this magical bit-shifting we aren’t accounting for visibility off the first rank. After all, if we are blocked forward 1 and not forward 2, this might still give that as a valid move. After all of this work on pawn movements, it might be better just to leave them to special cased code because of their intricacies compared to other pieces. What can we do then to quickly generate pawn moves? Some ideas are floating around for sure. If I have a BitBoard of all my pawns I can easily do the following:

uint validMoves = (curPawns << & (~(myPieces & enemyPieces)); // one sided, but basically advance and check.
validMoves |= ((validMoves & 0xFF0000) << & (~(myPieces & enemyPieces)); // all that were able to move forward 1, move 2

Some of the above information would be pregenerated, but you can see all of the gruesome details in action. Captures are a basic extension of the above. The basic premise involves 7 or 9 bit shifting operations along with clearing of the overflow file. What do I mean by overflow file? Well, take the rightmost file, h, and then tell me how a pawn would attack off the right side of the board. If we shift by 9 still the pawn in file h, would wrap around and attack the other side of the board. Clearing the pawns in the oveflow ranks is the only way I can think of getting rid of this for now.

Live Help Server: Jerry Messenger Server is Live Chat with Users on your websites.

Also see: REST2SQL in a Jiffy, with Tagspace for Spice

Also see: Resizing a Form has always been a pain in the rectum…

Also see: Reporting Services administration changes in Katmai (v.Next)

Also see: Brad Abrams’ pixel8 Interview Podcast posted

Also see: Big in Japan

Also see: Prototypes and Java Config with Spring

Also see: Yes, it does mean everything

Also see: Dare Obasanjo on C# Anonymous Types

Also see: Win friends and influence your team

Also see: Single source code base for Silverlight and WPF solutions

Also see: C# 3.0 Lambdas and Type Inference

Also see: Dare Obasanjo on C# Anonymous Types

Also see: TransparentProxy

Also see: Quaker votes

Also see: Doing the Deal and Dishing the Dirt

Also see: LoadFrom’s Second Bind

Also see: Never keep your emotions bottled up

Also see: LINQ - The Uber FindControl

Also see: Prototypes and Java Config with Spring

Also see: Hello world!

Also see: Bloggers in the Mavs Locker Room ?

Also see: Bloggers in the Mavs Locker Room ?

Also see: LINQ - The Uber FindControl

Also see: DevWeek 2008 Cross Platform Silverlight Demos

Also see: Memory Model

Also see: Generating WPF Content with LINQ

Also see: LINQ - The Uber FindControl

Also see: Bloggers in the Mavs Locker Room ?

Also see: Doing the Deal and Dishing the Dirt

Also see: Compatability

Also see: Win friends and influence your team

Also see: Never keep your emotions bottled up

Also see: Publishing: Good reviews, bad reviews, and hurting oooh so many feelings.

Also see: Big in Japan

Also see: Quaker votes

Also see: Never keep your emotions bottled up

uint validCaptures = ((curPawns & clearRank[0]) << 7) & enemyPieces;
validCaptures |= ((curPawns & clearRank[7]) << 9) & enemyPieces;

It is fairly nice to be able to calculate all pawn moves simultaneously in this manner. I’d still calculate en-passant separately because it has side-effects. The attacking square is different from the square where the enemies piece would be removed. It is an extra piece of information that the board has to carry around to make sure that it always performs the appropriate captures.

If you have questions about some of the rules, I actually found the following a bit helpful. I have a chess rules book that I tend to go to first for explanation, especially since I need to ensure accuracy in the engine. When I don’t feel like leafing I’ll check something out here in the FAQ and then put a little comment in the code to go back later and check the official rules http://www.chessvariants.org/d.chess/faq.html. Even if you know how to play chess, these FAQs can be algorist playgrounds, since often overlooked newbie questions can point out subtleties, patterns and optimizations you might not otherwise dream-up.

http://weblogs.asp.net/justin_rogers/archive/2004/10/24/246983.aspx

The Exception Model

Administrator — Mon, 24 Mar 2008 10:48:16 +0000

Also see: Spring Web Flow features and feedback request

I had
hoped this article would be on changes to the next version of the CLR which
allow it to be hosted inside SQL Server and other “challenging”
environments. This is more
generally interesting than you might think, because it creates an opportunity
for other processes (i.e. your
processes) to host the CLR with a similar level of integration and control. This includes control over memory usage,
synchronization, threading (including fibers), extended security models,
assembly storage, and more.

< ?xml:namespace prefix = o ns =
"urn:schemas-microsoft-com:office:office" /> size=2>

However,
that topic is necessarily related to our next release, and I cannot talk about
deep details of that next release until those details have been publicly
disclosed. In late October,
Microsoft is holding its PDC and I expect us to disclose many details at that
time. In fact, I’m signed up to be
a member of a PDC panel on this topic.
If you work on a database or an application server or a similarly
complicated product that might benefit from hosting the CLR, you may want to
attend.

size=2>

Also see: Exception Handling in Running a Business

Also see: Brad Abrams’ pixel8 Interview Podcast posted

Also see: LoadFile vs. LoadFrom

Also see: LINQ - The Uber FindControl

Also see: Spring Web Flow features and feedback request

Also see: The Internet is Officially Dead & Boring - Its the economy stupid !

Also see: Java design, operator overloading and people

Also see: DevWeek 2008 Cross Platform Silverlight Demos

Also see: Resizing a Form has always been a pain in the rectum…

Also see: Memory Model

Also see: TransparentProxy

Also see: Access to old blogs

Also see: Resizing a Form has always been a pain in the rectum…

Also see: Big in Japan

Also see: TransparentProxy

Also see: The Internet is Officially Dead & Boring - Its the economy stupid !

Also see: Tagspace, Meet Claimspace

Also see: TransparentProxy

Also see: I’ve finally settled into my new position on the Internet Explorer team…

Also see: LINQ - The Uber FindControl

After
we’ve disclosed the hosting changes for our next release, you can expect a blog
on hosting in late October or some time in November.

size=2>

Instead,
this blog is on the managed exception model. This is an unusual topic for me. In the past, I’ve picked topics where I
can dump information without having to check any of my facts or do any
research. But in the case of
exceptions I keep finding questions I cannot answer. At the top level, the managed exception
model is nice and simple. But – as
with everything else in software – the closer you look, the more you
discover.

size=2>

So for
the first time I decided to have some CLR experts read my blog entry before I
post it. In addition to pointing
out a bunch of my errors, all the reviewers were unanimous on one point: I
should write shorter blogs.

Custom Software Solutions. Billing and Invoicing Solutions, eCommerce and Website design.

Also see: Brad Abrams’ pixel8 Interview Podcast posted

Also see: ASP.NET MVC in CodePlex and Extensible Unit Testing

Also see: Reporting Services administration changes in Katmai (v.Next)

Also see: Hello world!

Also see: Spring Web Flow features and feedback request

size=2>

Of
course, we can’t talk about managed exceptions without first considering Windows
Structured Exception Handling (SEH).
And we also need to look at the C++ exception model. That’s because both managed exceptions
and C++ exceptions are implemented on top of the underlying SEH mechanism, and
because managed exceptions must interoperate with both SEH and C++
exceptions.

size=2>

Windows
SEH

size=2>

Since
it’s at the base of all exception handling on Windows, let’s look at SEH
first. As far as I know, the
definitive explanation of SEH is still Matt Pietrek’s excellent 1997 article for
Microsoft Systems Journal: http://www.microsoft.com/msj/0197/exception/exception.aspx . There have
been some extensions since then, like vectored exception handlers, some security
enhancements, and the new mechanisms to support IA64 and AMD64. (It’s hard to base exceptions on FS:[0]
chains if your processor doesn’t have an FS segment register). We’ll look at all these changes
shortly. But Matt’s 1997 article
remains a goldmine of information.
In fact, it was very useful to the developers who implemented exceptions
in the CLR.

Custom Software Development for Real-Estate, Hosting providers, Workflow and Business Management Systems.

size=2>

The SEH
model is exposed by MSVC via two constructs:

size=2>

style="MARGIN: 0in 0in 0pt; tab-stops: list.5in; mso-list: l0 level1 lfo1">__try {…}
__except(filter_expression) {…}
style="MARGIN: 0in 0in 0pt; tab-stops: list.5in; mso-list: l0 level1 lfo1">__try {…} __finally
{…}

size=2>

Matt’s
article explains how the underlying mechanism of two passes over a chain of
single callbacks is used to provide try/except/finally semantics. Briefly, the OS dispatches an exception
by retrieving the head of the SEH chain from TLS. Since the head of this chain is at the
top of the TIB/TEB (Thread Information Block / Thread Environment Block,
depending on the OS and the header file you look at), and since the FS segment
register provides fast access to this TLS block on X86, the SEH chain is often
called the FS:[0] chain.

Softwre Development for small and middle size companies. World-class software applications.

Also see: DevWeek 2008 Cross Platform Silverlight Demos

Also see: TransparentProxy

Also see: The Internet is Officially Dead & Boring - Its the economy stupid !

size=2>

Each
entry consists of a next or a prev pointer (depending on how you look at it) and
a callback function. You can add
whatever data you like after that standard entry header. The callback function is called with all
sorts of additional information related to the exception that’s being
processed. This includes the
exception record and the register state of the machine which was captured at the
time of the exception.

size=2>

To
implement the 1^st form of MSVC SEH above (__try/__except), the
callback evaluates the filter expression during the first pass over the handler
chain. As exposed by MSVC, the
filter expression can result in one of three legal values:

size=2>

EXCEPTION_CONTINUE_EXECUTION
= -1

Live Person Software: Turn website visitors into your customers.

Also see: Single source code base for Silverlight and WPF solutions

Also see: Exception Handling in Running a Business

Also see: A web site is not an RSS feed…nor the reverse.

Also see: Resizing a Form has always been a pain in the rectum…

Also see: ASP.NET MVC in CodePlex and Extensible Unit Testing

Also see: Compatability

Also see: Reporting Services administration changes in Katmai (v.Next)