среда, 15 сентября 2010 г.

Bad news, everyone!

If it is true..

update (18/09/2010/):
Microsoft has released an advisory to help customers understand the vulnerability and apply workarounds to secure their sites. The advisory is at http://www.microsoft.com/technet/security/advisory/2416728.mspx.

The Microsoft Security Response Center (MSRC) has released a blog at http://blogs.technet.com/b/msrc/archive/2010/09/17/security-advisory-2416728-released.aspx.

The Security Research & Defense (SRD) team at Microsoft has also released a blog that contains a script to help detect vulnerable installations. The blog is located at http://blogs.technet.com/b/srd/archive/2010/09/17/understanding-the-asp-net-vulnerability.aspx

Scott Guthrie about the vulnerability
http://weblogs.asp.net/scottgu/archive/2010/09/18/important-asp-net-security-vulnerability.aspx

This is really scaring me.. http://www.youtube.com/v/yghiC_U2RaM

вторник, 16 марта 2010 г.

C# iterators based scheduling

If you ask Google about asynchronous iterators you'll see that a lot of people have been using C# iterators for doing things that iterators were not supposed to do in the first place - from effective and convinient IO to microthreading and cooperative multitasking.
Several months ago being influenced by CCR iterators and AsyncEnumerator I tried to create microthreading library that could schedule microthreads on many OS threads and be integrated with different async patterns - from APM through EAP to TPL's Tasks. The early prototype looked promising but iterators are really optimzed for generating sequences effectively so there are some restrictions and inconveniences.
I hope the implementation of  resumable methods that may appear in future C# will let us build on top of it all aforementioned scenarios, and even those which the future methods are not supposed to be used for.

Why iterators were implemented the way they were implemented in Eric Clipperts' serie on iterator blocks.

воскресенье, 14 марта 2010 г.

C# 4.0 events. Small implementation change

C# 4.0 is going to change the way how event handlers addition and removal is implemented. From it birth C# compiler used to add special accessor methods to every event which didn’t have its own accessors declared. For thread safety those auto accessors had MethodImpl(MethodImplOptions.Synchronized) attribute defined on them. Using that attribute on publicly accesible methods is considered harmful because at runtime those methods are wrapped in lock(this) or lock(typeof([TypeDeclaringTheEvent])) for static events which, in its turn, is a bad practice as well.

In C# 4.0 event accessors don't use explicit locking, but implement what can be considered as optimistic locking. Look at the following:

//pre 4.0
[MethodImpl(MethodImplOptions.Synchronized)]
public void add_Disposed(EventHandler value)
{
    this.Disposed = (EventHandler) Delegate.Combine(this.Disposed, value);
}

// 4.0
public void add_Disposed(EventHandler value)
{
    EventHandler handler2;
    EventHandler disposed = this.Disposed;

    do
    {
        handler2 = disposed;
        EventHandler handler3 = (EventHandler) Delegate.Combine(handler2, value);
        disposed = Interlocked.CompareExchange(ref this.Disposed, handler3, handler2);
    }
    while (disposed!= handler2);
}

With the new algorithm we don't have to have one private object for each public event just to protect them from lockin issues. Besides that it is a little bit lighter and more performant. Delegate type itself hasn't changed at all. Delegates have been immutable reference types from the very begining. The only question I have: Why they didn't do that in the first place!?

More details in C# compiler team member's blog:
Field-like Events Considered Harmful
Events get a little overhaul in C# 4, Part II: Semantic Changes and +=/-=

воскресенье, 7 марта 2010 г.

What I’d like to see in C# 5.0

Axum, an experimental language, has such a feature as async methods. In such methods interaction with  interaction points and APM based API happens asynchronously.
Besides that there are signs that in the future C# will support resumable methods. Luca Bolognese opened up some details at the PDC 2009 session Future of C# and VB. Fast-forward to 50:00 where he shows experimental yield keyword in the method returning Async<T>.
That’s great news because with what we have now  if you want to make long running call not keeping your thread waiting/doing nothing/consuming  memory you have to put all your logic in callback methods calling each other. With resumable methods such code would be much  easier to write, read and refactor, and the exception handling would look like  synchronous, stack based.

But being excited about the feature:

  • I don’t like the idea of introducing a new type (Async<T>) when we already have Task and Task<T> types
  • I’d like C# to recognize not only APM pattern based API but methods and properties that return Task or Task<T> as well. If you compare Task and IAsyncResult you’ll see that a Task based asynchronous API is much simpler, cleaner and easier to implement. Compare the next two variations:

    //Task based API
    public Task  UploadData(string uri, string data)  { … }
    public Task<string> DownloadData(string uri) { … }
    public Task<int> TotalNumberOfRows { get { … } }

    //APM based API
    public IAsyncResult BeginUploadData(string uri, string data, AsyncCallback callback, object context)  { … }
    public void EndUploadData(IAsyncResult ar)

    public IAsyncResult BeginDownloadData(string uri, AsyncCallback callback, object context) { … }
    public string EndDownloadData(IAsyncResult ar)

    public IAsyncResult BeginGetTotalNumberOfRows(AsyncCallback callback, object context) { … }
    public int EndGetTotalNumberOfRows(IAsyncResult ar)

  • I’d like C# to recognize extension methods that implement APM or Task based API

суббота, 27 февраля 2010 г.

EqualityComparer for C# 3.0 Expressions

.NET Expression classes let us perform only reference equality checks on them. But sometimes we need to check the true value equality.  As part of my attempt to build autocacher for LINQ TO SQL compiled queries I created an equality comparer that implements IEqualityComparer<Expression> interface. It lets me, for example, to use an Expression as a key in a dictionary. The implementation is .NET 3.5 specific  and based on the example ExpressionVisitor class from MSDN. As a bonus  there are extensions to the Expression class in the project so that you can visit and rewrite  expressions without deriving from ExpressionVisitor. It can be very handy in simple scenarios.

ExpressionEqualityComparer.zip

p.s. BTW in .NET 4.0 they made  the ExpressionVistor class public.  In .NET 3.5 it  was internal.

среда, 10 февраля 2010 г.

LINQ TO SQL and asynchronous IO

LINQ TO SQL doesn’t inherently support asynchronous IO. It is unfortunate, because non-blocking IO is a way to get great scalability on server side whether you do it directly with APM,   CCR Iterators  and APM adaptors, Jeffrey Richter's AsyncEnumerator, TPL iterators'  integration with APM,  or even Axum which natively recognizes APM methods.

Well there is actually a small hole through which we can use APM but in a very restrictive way.
The best  tool to experiment with L2S is LinqPAD. Point it at Northwind database and try next “statement” query:

var query = from e in Employees
   where e.EmployeeID == 1
   select e;

System.Data.Common.DbCommand cmd = this.GetCommand(query);
try
{
 this.Connection.Open();
 System.Data.Common.DbDataReader reader = cmd.ExecuteReader();
 IEnumerable employees = this.Translate(reader);
 
 //using results
 Employees employee = employees.First();
 employee.Dump();
}
finally
{
 this.Connection.Close();
}

Instead of ExecuteReader we can use BeginExecuteReader and EndExecuteReader methods.  Wrap it in an convenient extension method for DataContext class and it seams we solved the problem. No.. Here is the list of problems we still have:

  • eager loading of many-to-one and one-to-many relations of the  entity in question doesn’t work with our method
  • to fulfil some queries L2S has to send more than one SQL commands to the server, but DataContext.GetCommand returns only the first one
  • L2S doesn’t support asynchronous lazy loading of  many-to-one, one-to-many  and Link<T> based properties of an entity
  • L2S doesn’t support asynchronous compiled queries
  • besides querying L2S doesn’t support  submitting changes asynchronously
  • and don’t forget about transactions

The list looks like a show stopper. But if you really  need to use APM with L2S and are not going to  eager load relations of the entity you want to query and sure that your  query will generate only one SQL command and won’t be surrounded by a transaction then you  should use it because parallelism and asynchronicity  is the way to performance and scalability.

воскресенье, 7 февраля 2010 г.

.NET threads and stack memory

Normally when an unmanaged thread is created 1 MB of virtual address space is reserved and 4 KB (a page) is committed for its stack. Reserved here is an important word, because reservation allocates no physical storage but only well.. reserves a range of virtual memory.  It guaranties continuousness of the memory required for threads’ stacks. Allocation of the real storage (committing) happens as the stack grows, page by page. 
But as address space in a process is usually limited by 2 GB  the more threads you create the less virtual address space is left. For example, a thousand of threads can consume a half of the available  virtual memory of the process. 
But recently I have found that the things are different in the managed world. When a managed thread created  1 MB of memory is reserved and committed for its stack before it starts. There is no difference between a managed thread and an unmanaged one regarding virtual memory space consuming but the early committing  brings the whole system closer to the commit limit which is “the sum of physical memory and the sizes of the paging files”.
It is one more reason not to create threads if you can. Use ThreadPool, Tasks or  async IO instead.

You can read more about the reasons behind the CLR team’s decision,
what Windows memory limits are and how  Processes and Threads consume memory.

среда, 27 января 2010 г.