Using the ALP Run-time library

The ALP Run-time library is nothing else but a set of COM objects your ASP pages can create and use through the standard ASP object creation facilities. They are just always available with ALP and you can count on their existence without additional efforts to gather re-distribution files, add new entries to the setup configuration, deal with OS compatibility issues and so on. These components are tested on all the platforms supported by ALP and they have minimal requirements. In fact they are even developed ahead of the ALP versions in order to allow us perform more testing (for example the ALP Run-time library is already available on Windows CE while ALP for CE is under development).

The previous chapter has shown in fact a little library usage - SQLite COM database. We will take a quick look at some other interesting features in this page. You can find more illustrations in the ALP samples and on the newObjects ActiveX Pack1 page. The ALP Run-time library is available also separately without ALP under the name newObjects ActiveX Pack1 family. This allows you transfer the library features to other environments. The samples shown on the newObjects ActiveX Pack1 family pages are mostly non-ALP samples, but they use the same objects you can use in ALP and the differences with the ALP use are minimal.

Storages and Files - using the file access components. 

Most of the ASP programmers associate the file access with the Scripting.FileSystemObject. There are many other components providing similar or even extended features. The ALP Run-time library contains several objects that cover its functionality and even much more. The general file access related objects in the library are:

SFMain - plays a role much similar to the FileSystemObject.
SFStream - provides text and binary access to streams and files.
SFStorage - provides access to storages and directories.
SFFileStream and SFDirStorage - implement low level stream and respectively storage interfaces to files and directories
SFFilter - implements low level binary access to streams and files
SFRecord and SFField - implement record based access over streams through a SFFilter object.
There are also some helper objects that hold file and storage information details etc.

In the most cases the ASP pages create directly only the SFMain object and they receive the other objects as a result of member calls. Lets take a look at a small piece of code that opens a text file and reads it line by line:

<%
  Set sf = Server.CreateObject("newObjects.utilctls.SFMain")
  Set file = sf.OpenFile(Server.MapPath("test.txt"),&H40)
  lineNum = 1
  While NOT file.EOS
    line = file.ReadText(-1) ' this is equivalent to the ReadLine method in FSO
    %>
      <%= lineNum %>: <B><%= Server.HTMLEncode(line) %></B><BR>
    <%
    lineNum = lineNum + 1
  Wend
%>

The SFMain.OpenFile method used in this code opens the test.txt file specified for reading only and returns an initialized SFStream object which is saved in the file variable by this example code. Internally the OpenFile method will do more than you can see from this code. When you request a file open operation a SFFileStream object is created and bound to the file, then a SFStream object is created and bound to that SFFileStream object. The library allows you skip these details in case of regular files, but knowing about them will help you understand the concepts. The role of each of these objects is: SFFileStream implements the standard IStream interface over a regular file, the SFStream on the other hand works through this IStream interface and is not particularly interested in the physical nature of the stream.

The benefit of this abstract mechanism is that it can be applied to non-file objects that behave like files. For example we can open a stream in an OLE storage file and read it with exactly the same code:

<%
  Set sf = Server.CreateObject("newObjects.utilctls.SFMain")
  Set stg = sf.OpenStorageFile(Server.MapPath("mystorage.stg"),&H40)
  Set file = stg.OpenStream("test.txt"),&H40)
  lineNum = 1
  While NOT file.EOS
    line = file.ReadText(-1) ' this is equivalent to the ReadLine method in FSO
    %>
      <%= lineNum %>: <B><%= Server.HTMLEncode(line) %></B><BR>
    <%
    lineNum = lineNum + 1
  Wend
%>

Obviously the different nature of the OLE storages requires us to use another path to the stream we are looking for - we need to open the OLE storage (OpenStorageFile) and then open a stream from it (OpenStream) as like the storage is a directory and the stream is a file. The rest of the code will be the same.

We can go even further - open a TCP connection and read it as text file:

<%
  Set nsMain = Server.CreateObject("newObjects.net.NSMain")
  Set addr = nsMain.GetHost("myserver.com")
  addr.Port = 8121 ' See the notes below
  Set socket = nsMain.NewSocket
  socket.Connect(addr)
  Set file = Server.CreateObject("newObjects.utilctls.SFStream") 
  file.SetStream socket
 
  lineNum = 1
  While NOT file.EOS
    line = file.ReadText(-1) ' this is equivalent to the ReadLine method in FSO
    %>
      <%= lineNum %>: <B><%= Server.HTMLEncode(line) %></B><BR>
    <%
    lineNum = lineNum + 1
  Wend
%>

Of course this code will make sense if the other side of the connection writes continuously. In this kind of usage EOS will return True when the socket is disconnected. The network connections are most often used for conversations - i.e. we send something and then we read the response, then we are repeating the cycle again. Still, the code used remains very similar to the code used with files, it is even possible to simulate network connection behavior using a test file.

The only significant difference between the different kind of streams you can use through the library components is the support for random positioning. For instance a file stream can be positioned at any location - i.e. you can start reading at N-th byte then move to the M-th byte and write something there. On the other hand the network streams cannot be positioned and can be read/written only sequentially. The term seekable is most often used to characterize the streams that can be randomly accessed and non-seekable is used for the streams that cannot. If a stream is seekable or not is most often obvious from its nature - obviously the network connection does not represent a block of persistent data while the files are stored in the file system and can be addressed like memory blocks.

There are certain library features applicable only for seekable streams and they wont work over non-seekable streams. A good example can be devised using another feature of the library - the record based access to files:

' Assume dataStrm is an already opened stream - a file or something else
Set rec = Server.CreateObject("newObjects.utilctls.SFRecord")
rec.AddField "Name", vbString, 31
rec.AddField "Age", vbLong
rec.AddField "Gender", vbBoolean

rec.BindTo dataStrm 
rec.Filter.unicodeText = False
rec.MoveFirst 
Dim recCount 
recCount = rec.RecordCount

While Not dataStrm.EOS %>
  <HR>
  Name: <B><%= rec("Name").Value %></B><BR>
  Age: [<%= CLng(rec("Age").Value) %>]<BR>
  Gender: 
  <% If rec("gender").Value Then 
    Response.Write "Male"
  Else 
    Response.Write "Female" 
  %><BR><%
  rec.MoveNext
Wend

This piece of code will work correctly over a file - i.e. if the dataStrm is opened like in the samples above. Here we create a SFRecord object and add a few fields in it. The AddField method automatically creates a SFField object and initializes it, thus it is just a quick way to define a record structure. The same can be done by creating SFField objects directly, initializing them and adding them to the record, but it will require more code.

The BindTo method binds the SFRecord object to the stream. In the documentation you will notice that this method has one more optional parameter where a SFFilter object can be passed. However the BindTo method automatically creates a SFFilter object if none is passed explicitly. Most often you need to specify some general details about the data format in the stream - others than the field types and sizes. In this case we specify that the text values (i.e. the fields that contain strings - in this sample this is the Name field) are NOT UNICODE. The role of the filter objects is to stay between the SFRecord object you use and the stream itself. The SFFilter drives the low level communication with the stream and caches the records. It can be also used directly without a SFRecord by applications requiring lower level access (You can see this done in the Executable file information sample from samples installed by ALP).

When this is done we can ask the SFRecord to count the records in the stream, then we can cycle through the stream and read the records one by one. This technique allows us to define the structure of the records - in this sample we assume that the stream contains records with 3 fields each: Name, Age and Gender. The sizes of the Age and Gender fields is defined implicitly by their types, but the size of the Name field must be specified (31 characters in our sample). Further the interface looks very similar to a database recordset - we use SFRecord's MoveNext method  to move to the next record, and we can use other Move methods to position randomly if this is needed - for instance rec.Move 10, 0 will position the SFRecord over the 10-th record in the stream.

Now lets return to the seekability. Note that nor RecordCount, nor MoveNext/MoveFirst or Move can function without seekability. To count the records we need to know the current size of the stream, to move randomly we must be able to position randomly in it. The SFRecord, knowing the record structure just makes the calculations for us and repositions the stream. Apparently this can be done over a file or a stream from an OLE storage file or in a memory stream, but if we use a network connection for example there is no way to know the size of data that will be received, nor we can position over it. That is why some of the objects in the library support alternative ways to read/write records. For instance SFRecord has the Read and Write methods which will read/write the record at the current position of the stream, but will not attempt any repositioning. In case of a file this will mean that after reading a record you are positioned over the next one, in case of a network connection the data in the record is just sent. Therefore the above sample code can be changed this way to work without requiring the stream to be seekable:

' Assume dataStrm is an already opened stream - a file or something else
Set rec = Server.CreateObject("newObjects.utilctls.SFRecord")
rec.AddField "Name", vbString, 31
rec.AddField "Age", vbLong
rec.AddField "Gender", vbBoolean

rec.BindTo dataStrm 
rec.Filter.unicodeText = False
rec.MoveFirst 
Dim recCount 
recCount = 0

While rec.Read %>
  <HR>
  Name: <B><%= rec("Name").Value %></B><BR>
  Age: [<%= CLng(rec("Age").Value) %>]<BR>
  Gender: 
  <% If rec("gender").Value Then 
    Response.Write "Male"
  Else 
    Response.Write "Female" 
  %><BR><%
  recCount = recCount + 1
Wend

As the Read method returns a Boolean result indicating success we can use it instead of EOS (End Of Stream) property, but of course EOS can be used if we prefer that. We can count the records only by incrementing the variable recCount while we read them, but not before we read all the records from the stream, because the size is not known if the stream is non-seekable.

The seekability concerns some other features as well. For example the ReadText method we used in some samples above has an argument that specifies how many characters to read. The negative numbers of -1, -2 and -3 have special meanings instructing the method to determine their number over a rule. For instance -1 will read up to the end of a text line where the end of the text line is recognized as a sequence of characters as specified in the SFStream's textLineSeparator property, the -2 will read the entire stream, the -3 is much like -1 but it will recognize the end of the text line even if the line separator characters are in unexpected order. For example in Windows the text lines end with <CR><LF> characters, but if you read a text file is UNIX formatted the text lines will end with <LF> character only. Thus the -3 value instructs the ReadText method to attempt all the permutations of the characters specified in the textLineSeparator property. However, this may require the ReadText method to re-position the stream after determining the end of the line, so it would remain at the beginning of the next line after the operation. Thus the -3 option of ReadText requires seekable streams. These requirements are specified in the documentation and you must notice them when you are going to work with non-seekable streams.

We used numeric literals in the samples above, but you can include the constants definition and use friendly names instead of the numbers. While ReadText has only 3 constants which if frequently used can be remembered, the OpenFile/OpenStream flags are many and using the named constant definitions will make the code much readable.

Threads

Many ASP developers know that there are tasks that require more time to complete than an ASP page will permit. Even if the time is not too long you always risk the user to decide that the application has failed and close it or stop the navigation thus interrupting the work you want to finish. Sometimes it is even worse - you may need to establish a code that will perform service tasks scheduled by the application and there is no way to do so using a regular ASP page. Even in non-WEB development environment (like VB for instance) such a task will need to be performed in the background in order to keep the user interface responsive and not block it completely while the task is in progress. This is where the threads come handy. Lets take a look at a simple example:

1. The code in an ASP page that creates the thread:

If Not IsObject(Application("Thread1")) Then
    Set Application("Thread1") = Server.CreateObject("newObjects.utilctls.COMScriptThread.free")
End If
Set thread = Application("Thread1")

Message = ""

' Initialize the max number for user convenience
If Request("MaxNum") = "" Then 
    MaxNum = 30000
Else
    MaxNum = CLng(Request("MaxNum"))
End If


' Configure the thread's parameters
thread.Value("MaxNum") = CLng(Request("MaxNum"))
thread.Value("CurNum") = CLng(0)
thread.Value("Found") = CLng(0)
thread.Value("Results") = ""
                        
Set sf = Server.CreateObject("newObjects.utilctls.SFMain")
Set file = sf.OpenFile(Server.MapPath("threadscript.vbs"),&H40)
        
' Start the thread.

If Not thread.Start("VBScript",file.ReadText(-2)) Then
    ' An error has occured.
    Message = "Error starting the thread: " & thread.LastError
Else
    Message = "Thread has been started."
End If

2. The code running in the thread (the threadscript.vbs file we load in the code above).

Function IsPrime(n)
    Dim j
    For j = 2 To n / 2
        If n Mod j = 0 Then
            IsPrime = False
            Exit Function
        End If 
    Next
    IsPrime = True
End Function

For I = 0 To Context("MaxNum")
    Context("CurNum") = CLng(I)
    If IsPrime(I) Then
        Context("Found") = Context("Found") + 1
        If Context("Results") <> "" Then Context("Results") = Context("Results") & ", "
        Context("Results") = Context("Results") & I
    End If
Next

3. And a code in an ASP page that examines the current thread progress or result if it has finished.

Set thread = Application("Thread1") 
<% If thread.Busy Then %>
  <B>Still Running</B><BR>
  Current number <B><%= thread.Value("CurNum") %></B><BR>
<% Else %>
    <% If IsEmpty(thread.Value("MaxNum")) Then %>
        <B>Has not been started yet.</B><BR>
    <% Else %>  
        <% If thread.Success Then %>
            <B>Finished Successfuly</B><BR>
            <%= thread.Value("Results") %><BR>
        <% Else %>
            <B>Error occured</B><BR>
            <B><%= thread.LastError %></B><BR>
        <% End If %>
    <% End If %>
<% End If %>

The thread in the sample code seeks for prime numbers in quite an ineffective way in order to take more time and make the sample substantial.

The first important step done by the page that starts the thread (1) is to create the COMScriptThread object or use an existing one if it is already created. Special attention deserves the fact that the COMScriptThread object is saved in an Application variable. Without this running a thread will make no sense, because after the ASP page completes the contact with the thread will be lost. Thus in the "real world" the application should do even more if the thread object is already created - check if it is in use and if that is so delay the new task for example, or may be create another COMScriptThread object. We will skip these details here, but you can take a look at the Script thread example in the samples set installed with ALP to see a more precise implementation.

The next important step are the parameters through which the ASP pages will communicate with the thread. COMScriptThread supports a collection of variables similar to the Application and Session in ASP. This collection is accessible for the both ASP pages and the script that runs in the thread. The "outer world" - the ASP pages access it using the Value property of the COMScriptThread object. You can see the code 1 setting several named values there. On the other side - in the script that runs in the thread the same collection is visible through a global namespace Context -see the code segment 2 above. The name is different but the values are the same - so the both sides can exchange information, the ASP pages can pass parameters that will alter the thread behavior etc. This collection is most often called Thread context in the documentation and the examples. COMScriptThread uses another object from the run-time library to implement it - the VarDictionary object. Therefore the Value property and the Context namespace expose the same VarDictionary object for the both parties. This implies that you can use over that object the methods and properties listed in the VarDictionary documentation. For instance you can obtain the count of the values in the collection in the ASP page by using thread.Value.Count or on the other side (in the thread) by using Context.Count.

By default the Thread context is configured to refuse objects. Thus by default in result of an attempt to set an object in it you will receive an error message, but passing objects as parameters is not impossible. To do so you must change the Thread context behavior by setting thread.Value.extractValues = False (see VarDictionary in NDL). The HTTP server sample (installed by ALP) uses this feature because the ASP controller page in the example wants to be able to interrupt the HTTP server by closing its listening socket which is kept in the Thread context. The default behavior prevents objects to be passed as parameters in order to help avoid some human mistakes. It is not possible to pass any object as parameter to a thread. The developer must be aware of the object capabilities and life time before considering passing it as parameter. For example passing the ASP Request object as parameter will be wrong, because it will be no longer functional after the ASP page finishes its execution. This may be obvious, but there are other cases in which it is not so obvious - consider passing an ADO recordset which is closed later in the ASP page - this will be wrong again because the thread may try to access it after that moment. Thus passing objects to the thread requires you to know well how they behave, if they depend on the state of other objects, if their life-time will be enough and so on. Thus the default behavior guarantees that only basic values will be recorded in the Thread context and no issues will occur. This is especially useful in JScript where no equivalent of the VBScript's Set operator exists and one can pass an object in a mistake. In this case the collection will assume that the developer meant to pass a value and will attempt to extract it from the default property of the object (if it supports such).

Finally in the code segment 3 above we are inspecting the thread's state and display it. We can check if the thread is still busy (see the Busy property)  performing the task or is idle. If it is not doing anything and we are sure we have started it it is logical to check if an error has occurred in it and display it - see the lastError and the Success properties..

The applications can do more with the threads. Some of the typical usages are: Passing a database connection to delegate to a thread a long lasting data mining operation; Creating a set of threads (thread pool) to implement efficient server software; Performing network operations such as fetching/sending e-mail in the background while the user reads the already received messages using the other ASP pages in the application; Implementing application automation tasks such as generating, importing/exporting from/to office documents and so on. Generally any task that requires none or only occasional user attention before it is finished fits best in a thread. The ALP applications thus are able to implement user interface which will never get stuck while the application is busy performing the requested operations - which is one of the most unnerving issues for the most users.

Quick look at other run-time library features

StringUtilities object (newobjects.utilctls.StringUtilities) provides string formatting functionality which will be familiar for all the developers who know about the printf/sprintf functions in the C standard libraries. In ASP and other scripting environment it is something one often misses and the library fills this gap. However, there are additional opportunities to a such implementation in a COM environment such as ASP pages and the StringUtilties object implements many extensions to the format specification known from C preserving the backward compatibility. For example there are the %q and %Q escapes which will format a value to string escaping the quotes in it so that it can be placed directly in an SQL statement. The StingUtilities' SCprinf and SAprintf methods in contrast to the Sprintf method do not use variable arguments count. Instead they are fetching the arguments from a collection object or from an array. This allows also an extension that allows you refer to the arguments non-sequentially but by name or index in collection/array. The object also supports custom date/time formatting which can be controlled directly by the application no matter if the operating system has the locale specific files installed. The automatic formatting allows you specify %a escapes which will try to format the argument to one of the several formats you specify in a separate property, thus giving you opportunity to gain more flexibility with arguments that may hold different value types. There is also support for NULL values which is also handy for SQL statements.

VarDictionary and UtilStringList are universal collection objects. VarDictionary deserves more attention - it can be compared to the Scripting.Dictionary you may know from the your experience. Still this object is more advanced and supports features that allow effective code to be created for more complex data structures. For instance if you put VarDictionary objects as values in another VarDictionary object you can construct a tree in the memory. As it grows larger you will feel the need of a feature that will allow you search through it in depth. VarDictionary offers methods like FindByValue, FindByName and FindByInfo. They perform a serch in depth and return set of references to the "nodes" that match the criteria. Such a feature allows you use VarDictionary to create structures that offer features similar to the XML object model and often they will be simpler to use than XML. VarDictionary also offers a Clone method that allows you clone the object with its contents (even if it is a tree as described above), the behavior properties extractValues, itemsAssignmentAllowed, readOnly and so on allow you adjust the object behavior to match your particular needs or prevent wrong usage in other parts of the application. The Info, Missing and Root properties allow metadata to be attached and further behavior refining - for example the Missing property specifies what will be returned when the application attempts to read a non-existent value.

Thus the features of the VarDictionary object make it convenient for various purposes. Many other library objects use it internally, some of them return it through some of their properties or methods. This allows unification of the data obtained from different objects using the common VarDictionary features. Knowing this may save you some coding efforts in many places by giving you the opportunity to transfer or otherwise organize the data. Object that use VarDictionary: ConfigFile, SFStorage, SFMain, COMScriptThread, ScriptManager, VaryDisp, SQLiteCOM and so on.

SQLite COM. This object with little help of the VarDictionary object implements alone a fully functional SQL database engine. It is based on the SQLite free source code, but also adds some features convenient in Windows environment. The database engine and the interface to it are both in this single object, it does not use ADO, OLEDB or MS Jet (the MS Access' database engine) and is thus independent of what you have installed on your machine. It uses non-recordset based database interface which will look familiar and convenient to the ADO developers who like the usage of ADO Recordset's GetRows method. Still, SQLite COM interface is something between a recordset and array caring the features that require less code to implement database functionality. A typical database operation with SQLite COM requires up to 2 times less code than the same operation performed with ADO, the number of methods and properties you need to learn is very low (about 10 from which only 3-4 are frequently used).

The actual benefit of using SQLite COM is obvious if you consider the fact that it uses single file to store all the database objects, the database file format is device independent (it can be copied to a Pocket PC and used there without need of conversion), there is no need of external configuration, nor need to perform any preliminary steps prior to using the database - it can even run in pure autorun scenarios where the component is not installed. Therefore besides the obvious applications such as programs that aim to ultimate compatibility, autoruns that must run directly from a removable media there are other applications for SQLite COM. For example consider an application that works mainly in your organization's infrastructure, it uses access to internal database servers, office applications and other specific software. Sometimes a need may arise to create an application that must work with some data extracted from the local database servers and applications, but it must work separately on machines not connected to your network. Another application is the case in which you may need to export information extracted from such an environment to a file and use/transfer it to another office or organization. You can maintain a file or set of files, but this will require you to write additional code to support their structure - very often one dreams for a database interface to such temporary or permanent export files - SQLite COM databases are stored in single file binary compatible with all the platforms so you do not need to go for more complicated solutions if a database interface fits your needs.