Friday 18 November 2011

Process.WaitForExit (Int32) hangs problem

I recently encountered this problem while using a method I use to call commands from the shell via the cmd.exe process.

The code looks like almost this:



code 1:
   1 using System;
   2 using System.Collections.Generic;
   3 using System.Linq;
   4 using System.Text;
   5 using System.Diagnostics;
   6 
   7 namespace TestWaitForExit
   8 {
   9     class Program
  10     {
  11         static int Main(string[] args)
  12         {
  13             CommandResult Result = ExecuteShellCommandSync(@"ping -t 127.0.0.1", 1000);
  14 
  15             return 0;
  16         }
  17 
  18         public static CommandResult ExecuteShellCommandSync(String CommandString, int Timeout)
  19         {
  20             CommandResult ToReturn = new CommandResult();
  21 
  22             Process process = new Process();
  23         
  24             System.Diagnostics.ProcessStartInfo startInfo = new System.Diagnostics.ProcessStartInfo();
  25                     
  26             startInfo.WindowStyle = System.Diagnostics.ProcessWindowStyle.Hidden;
  27                     
  28             startInfo.UseShellExecute = false;
  29             startInfo.RedirectStandardError = true;
  30             startInfo.RedirectStandardOutput = true;
  31         
  32             startInfo.FileName = "cmd.exe";
  33         
  34             String MyCommand = @"/C " + CommandString;
  35        
  36             startInfo.Arguments = MyCommand;
  37 
  38             process.StartInfo = startInfo;
  39         
  40             process.Start();
  41 
  42             String Output = "";
  43             String Error = "";
  44             try
  45             {            
  46                 Output = process.StandardOutput.ReadToEnd();
  47                 Error = process.StandardError.ReadToEnd();
  48             }
  49             catch
  50             {
  51                 Output = "Can't read stdout";
  52                 Error = "Can't read stderr";
  53             }
  54 
  55             process.WaitForExit(Timeout);       
  56                     
  57             int Exit = 0;
  58 
  59             if (process.HasExited == true)
  60             {
  61                 ToReturn.ProcessNotExited = false;
  62                 Exit = process.ExitCode;
  63             }
  64             else
  65             {
  66                 ToReturn.ProcessNotExited = true;
  67             }
  68         
  69             process.Dispose();
  70 
  71             ToReturn.ReturnedCode = Exit;
  72             ToReturn.OutputString = Output;
  73             ToReturn.ErrorString = Error;
  74 
  75             return ToReturn;        
  76         }
  77     }
  78 
  79     public class CommandResult
  80     {
  81         public int ReturnedCode { get; set; }
  82         public String OutputString { get; set; }
  83         public String ErrorString { get; set; }
  84         public Boolean ProcessNotExited { get; set; }
  85     }
  86 }

As you can see, the code starts a "cmd.exe" process and passes to it the command I want to be executed.


I redirect StandardError and StandarOutput in order to read them from the code.
The code reads them before the process.WaitForExit(Timeout) call as recommended by Microsoft (more on this later).
The problem arises if the command I send to "cmd.exe" never terminates or hangs indefinitely.
In the code I used the command ping -t 8.8.8.8 which, because of the -t option, pings the host without stopping. What happens? The "cmd.exe" process along with the ping -t command never exits and never closes the stdout stream and so our code hangs at the Output = process.StandardOutput.ReadToEnd(); line because it can't succeed reading all the stream.


The same happens also if a command in a batch file hangs for any reason and so the above code could work continuously for years and then hang suddenly without any apparent reason.

Before I wrote that it's recommended to read redirected streams before the process.WaitForExit(Timeout) call, well this is especially true if you use the WaitForExit signature without the Timeout.

If you call process.WaitForExit() before reading the redirected streams:

code 2:
   1 process.Start();
   2 process.WaitForExit();  
   3 
   4 String Output = "";
   5 String Error = "";
   6 
   7 try
   8 {                
   9 Output = process.StandardOutput.ReadToEnd();
  10 Error = process.StandardError.ReadToEnd();
  11 }
  12 catch
  13 {
  14 Output = "Can't read stdout";
  15 Error = "Can't read stderr";
  16 }

 you can experience a deadlock if  the command you attach to "cmd.exe" or the process you are calling fills the standard output or standard error. This because our code can't reach the lines
Output = process.StandardOutput.ReadToEnd();
     Error = process.StandardError.ReadToEnd();.
As a matter of fact the child process (the ping command or a batch file or whatever process you are executing) can't go on if our program doesn't read the filled buffers of the streams and this can't happen because the code is hanging at the line with process.WaitForExit() which will wait forever for the child project to exit.


The default size of both streams is 4096 bytes. You can test this two sizes with these batch files:

code 3:
   1 @ECHO OFF
   2 for /l %%X in (1,1,409) do ( echo|set /p=0123456789)
   3 echo 0123

code 4:
   1 @ECHO OFF
   2 for /l %%X in (1,1,409) do (echo|set /p=0123456789)  1>&2
   3 (echo 0123) 1>&2

The first script writes 4096 bytes to standard output and the second to standard error.

Save one of these into "C:\testbuffsize.bat" and run our program calling process.WaitForExit() before 
Output = process.StandardOutput.ReadToEnd();
     Error = process.StandardError.ReadToEnd();
as in code 2, you can do it writing
CommandResult Result = ExecuteShellCommandSync(@"c:\testbuffsize.bat", 1000);
at line 13 of code 1.

 The code won't hang but if you write one more byte in any of the two streams it will overflow the buffer size making the program hang.

If you need to redirect and read the standard output or standar error the best solution is to read them asynchronously. An excellent way to do this is proposed by Mark Byers in this stackoverflow thread

As the last thing please notice that if the child process exits only because you use the process.WaitForExit(Timeout) signature and it actually goes in timeout you should kill the "cmd.exe" process and its possible children.

Saturday 11 June 2011

An easy circuit to transform a led/light pulse into a useful voltage for a pulse counter

I'm gonna show an easy (and probably not optimal) circuit to take the blink of a led and transform it into an electric signal that will become the logic state to be counted by a pulse counter device.

The counter device is a "SENECA Z-D-IN". This digital input device is able to count electric pulses and it is queryable with the modbus protocol.

Here is the circuit:

These are the components I used and the links to the data sheets:
  • diodo zener 8.2 V http://docs-europe.electrocomponents.com/webdocs/0de2/0900766b80de2c96.pdf
  • phototransistor BPX 43 http://docs-europe.electrocomponents.com/webdocs/0d00/0900766b80d00198.pdf
  • transistor BC107B http://docs-europe.electrocomponents.com/webdocs/0e77/0900766b80e779e3.pdf
  • opamp OPA251 http://docs-europe.electrocomponents.com/webdocs/0c8c/0900766b80c8ccc9.pdf
The 16 volt power can be supplied by the Seneca digital input counter.

I have used a phototransistor that must be attached to the pulsing led, maybe with some adesive tape so that it will remain in the dark when the led is off.

The phototransistor must be sensible to visible light and cannot be an infrared one. The Bpx is sensible to all the visible spectrum.

The 8.2Volt Zener diode is useful to maintain the voltage between the collector and emitter of the phototransistor less than 3V. This is because many phototransistors have a small maximum Vce and with this circuit the phototransistor will switch on a second transistor that can have higher maximum ratings.

The opamp is used as a subtractor of the voltages across tre resistor marked with "R5". So the output will be 0 V when the led is off and almost 16V when the led is on.

The resistors involved with the opamp are 1MOhm because I don't want current circulating across the "R6" and "R9" resistors.

The zener diode used is very small and must be soldered even if you use a breadboard.

This circuit has been tested with the Seneca device.


Sunday 27 February 2011

Postgresql: concurrent deletes made my application hang

Some days ago I noticed that occasionally (once every 2-3 weeks) my multithreaded .net console application was hanging .

I was sure that the issue was caused by a programming error since it's very easy to do something wrong when using threads.

Then, looking at the Postgresql log file, I saw that my program was causing a lot of deadlocks.

The deadlocks were caused by a trigger that is activated every time the software makes an INSERT into a specific table. This trigger deletes all the rows that are older than 2 days.

Since there are concurrent INSERTs (because the application works with more threads and since the application itself is running in many instances on different machines) also the DELETEs are concurrent.

To avoid the problem I had to delete the rows in a consistent order (see the postgresql documentation about deadlocks http://www.postgresql.org/docs/8.2/interactive/explicit-locking.html) in this way:
FOR row_to_delete IN SELECT * FROM table
WHERE row_time < NOW() - interval '2 day' ORDER BY id
LOOP
DELETE FROM table as A WHERE A.id = row_to_delete.id;
END LOOP;
Now that the DELETEs happen in a consistent order (id field) I didn't experience deadlocks anymore and the application never hanged again.

However Postgresql manage deadlocks aborting one of the transactiona involved when a deadlock occurs and so my application should not have been hanging also without the change in the trigger.
So I am not sure if the problem is 100% resolved or if Npgsql (the provider I use to access Postgresql) has some difficulties in managing deadlock situations.

Silverlight and Wcf: custom property setters and prevent serialization of certain fields in generated classes

In this article I will show you how to obtain custom setter (and getter) methods and to avoid serialization of certain properties that are only required in the client part of your application.

This article is based on the Pete Brown's work:
so I suggest to read that article before this one.

The tecniche shown in the Pete's article allows me to customize the code in the getters and setters because in the Silverlight client code I will be using the Silverlight version of the class.

Let's see the class that we want to serialize down the wire:
   1 using System;
2 using System.Net;
3 using System.ComponentModel;
4
5 namespace MyClassLibrary
6 {
7 public partial class ClassToBeSerialized : SerialObject, INotifyPropertyChanged
8 {
9 private String _Name = "";
10 public String Name
11 {
12 get
13 {
14 return _Name;
15 }
16 set
17 {
18 _Name = value;
19
20 NotifyPropertyChanged("Name");
21 }
22 }
23
24 #region "PropertyChanged"
25
26 public event PropertyChangedEventHandler PropertyChanged;
27
28 private void NotifyPropertyChanged(String info)
29 {
30 if (PropertyChanged != null)
31 {
32 PropertyChanged(this, new PropertyChangedEventArgs(info));
33 }
34 }
35 #endregion
36 }
37 }


This class is edited in a Silvelight class library project. As explained in the Pete's article this class is also added as a link in a full Clr class library.

While the code of the web service will use the full Clr version of the class, the client will use the
version build with the Silverlight assemblies. And we tell to Silverlight to use its own version of the class setting the flag "Reuse Types in referenced assemblies" in the web service reference configuration.

In this way we don't lose the get and set sections of our properties that would have been lost if we would have choosed to don't reuse the types in the referenced assemblies.

Now we want to avoid the serialization of certain fields because we will use them only in the client code and we don't want to download or upload them by the wcf web service and waste bandwidth.

To achieve this we can extend the partial class ClassToBeSerialized in the Silverlight class Library in a different file but, this time, we won't link this file in the full Clr project.

Doing in this way the web service code won't see the properties added in the class extension and so these properties won't se serialized.

Let's see the partial class extension:

   1 using System;
2 using System.Net;
3 using System.Windows;
4 using System.Windows.Controls;
5 using System.Windows.Documents;
6 using System.Windows.Ink;
7 using System.Windows.Input;
8 using System.Windows.Media;
9 using System.Windows.Media.Animation;
10 using System.Windows.Shapes;
11 using System.ComponentModel;
12
13 namespace MyClassLibrary
14 {
15 public partial class ClassToBeSerialized : SerialObject, INotifyPropertyChanged
16 {
17 // if you add things that you want to see only within the Silverlight Client
18 // you can add them in an other partial class file.
19 // in this way the compiler doesn't complain about the System.Windows.* namespace
20 // and so you can use, for example, things like a MessageBox
21
22 private String _Surname = "";
23 public String Surname
24 {
25 get
26 {
27 return _Surname;
28 }
29 set
30 {
31 _Surname = value;
32
33 _Surname += " is a nice Surname!!!";
34
35 MessageBox.Show("Surname Changed!");
36
37 NotifyPropertyChanged("Surname");
38
39 }
40 }
41 }
42 }
43

You can see that I alter the Surname every time I set it adding " is a nice Surname!!!". This just to show that when I modify this property in the client code the setter is enabled and working.

You can also note that I can use a MessageBox because I can use the System.Windows.* namespaces since this file is not linked in the full Clr project and so it's not cross compiled.

Here I link a small project where you can see these few things together:

You can verify that the Surname isn't serialized or sent across the wire using Fiddler. Since Fiddler won't display localhost traffic you may want to follow these steps:
1) Start the project by pressing F5 (start debug).
2) Copy the address from your browser address bar.
3) Stop the debugging session.
4) Right click the web project in the Visual Studio Solution explorer panel.
5) Select Property Pages -> Start Options
6) Select Start url and paste the url copied from the address bar and add a dot after localhost.
You should obtain something like this
http://localhost.:<your port> /SilverAssemblySerialization.Web/SilverAssemblySerializationTestPage.aspx
7) Change the web service reference configuration in the Silverlight client project to point to the right url (please note the dot):
http://localhost.:<your port> /SilverAssemblySerialization.Web/Service.svc
With these steps Fiddler will monitor the traffic of the application and you will avoid cross domain exception since the wcf service and the Silverlight client binary share the same domain http://localhost.:<your port>

The application should have enough comments tho show what is going on.