delphi3000.com - the free delphi knowledge platform
delphi3000.com - the free delphi knowledge platform
Have a look at your member-status

connecting people's knowledge


  - Recent ArticlesRSS feed for Recent Articles on delphi3000.com
  - List of All Articles
  - Top Viewed Articles
  - Articles (+Attachem.)
  - Articles Of Interest
  - Categories
  - Top Uploader
  - Search
  - Index

  - My Home
  - Submit an Article
  - My Articles
  - My Personal Data
  - My Bookmarks
  - Activities
  - Login/Logout

  - Sign Up
  - Why Sign Up
  - Newsletter

  - Press
  - Advertise

  - Contact
  - Feedback



Community
Borland
ClubeDelphi
Dr. Bob
UK-BUG
Delphi Meetings
Planeta Delphi








Share this article with friendsShare this article with friends
Rate this articleRate this article - to keep the quality of delphi3000.com !
Comment this article or read through previous comments (8)


Simple high performance 3-tier apps with Indy and ADOFormat this article printer-friendly!Bookmark function is only available for registered users!
Product:
Delphi all versions
Category:
Multi-Tier
Skill Level:
Scoring:
Last Update:
05/01/2002
Search Keys:
delphi delphi3000 article borland vcl code-snippet middleware 3-tier Indy ADO OLE-DB TCP/IP client/server
Times Scored:
9
Visits:
14148
Uploader: aka aka
Company:
Reference: N/A
 
Question/Problem/Abstract:
Now it's easy to create simple high performance middleware applications with Indy and ADO (thanks to OLE-DB resource poooling).
Answer:



First of all sorry for my bad english.

You can read the post of Romeo Lefter (seems to be romanian like me) on this site about this problem, but I'll use a diffrent aproach.

I'll skip the intro and bla..bla part (thanks to Romeo). What we need: Indy (www.nevrona.com/indy, VCL comps for easy TCP/IP communications) kbmMemtable (www.components4developers.com, advanced in-memory table), ADOExpress (Ado component suite included in last version of Delphi and CB) or Adonis (which I prefer). These examples are based on Adonis but are very easy to convert to AdoExpress (the diffrences between them are: ADOExpress has a component called AdoConnection; Adonis has AdoDatabase; the same thing but with diffrent name).

We'll use the same aproach as Internet Explorer: connect/send request/receive responese(data)/disconnect. This model is very scalable and very efficient.

We have following tiers: the client named CL (developed by us), middleware server named MD (developed by us) and database server named DB (can be any SQL database which has a OLE-DB provider with resource pooling - MS SQL SERVER, Oracle, Interbase(http://www.lcpi.lipetsk.ru/prog/eng/, Lipetsk provider)).
What is "resource pooling": you can connect to a db server thru a OLE-DB provider; the second connection to db server is very quick because the first call creates a pool of connections available for this process.
This example is based on MS SQL Server.

CL connects MD (thru Indy), CL sends to MD the sql query/statement, MD connects DB (thru ADO - very quick thanks to resource pooling), DB executes the request and sends response to MD, MD closes the connection to DB and sends the response to CL, which closes the connection to MD. In case of errors an exception is send back.
You can do custom RPC (remote procedure call). An example: when the CL sends the sql query/statement, the statement can look somethink like this: "@SEND_MAIL", and the MD executes a custom code (in this case sending mails) and not pass to DB (because it has @ in front of the string).

The use of TidThreadPoolMgr is very useful because the threads are maintained in a pool and can be used for thousands of requests.
A normal thing to do on MD server is to create dinamically (for every request) a ADOConnection or ADODatabase, a ADOQuery, a kbmMemtable, a VCLZip component if your response to be sent zipped.
To improve the performance a little and the quality of code we create a TSQLThread class with BeforeExecute and AfterExecute procedures to override the standard way (here we can create our components - only once at creation and destruction of thread). YES, the BeforeExecute and AfterExecute are executed once: at the creation and destruction.

MD server:
-------------------------------------------------------------------------
unit uMain;

interface

uses
  Windows, Messages, SysUtils, Classes, Graphics, Controls, Forms, Dialogs,
  IdThreadMgr, IdThreadMgrPool, IdBaseComponent, IdComponent, IdTCPServer,
  ADODS, Db, kbmMemTable;

type
  TSQLThread = class(TIdPeerThread)
  protected
    FADODb: TADODatabase;
    FADOQry: TADOQuery;
    Fmt: TkbmMemTable;
    procedure BeforeExecute; override;
    procedure AfterExecute; override;
  public
  end;

  TForm1 = class(TForm)
    TCPServer: TIdTCPServer;
    IdThreadMgrPool: TIdThreadMgrPool;
    procedure FormCreate(Sender: TObject);
    procedure FormDestroy(Sender: TObject);
    procedure TCPServerExecute(AThread: TIdPeerThread);
  private
    { Private declarations }
  public
    { Public declarations }
  end;

var
  Form1: TForm1;

implementation
uses ActiveX, ComObj;

{$R *.DFM}

procedure TSQLThread.BeforeExecute;
begin
   CoInitialize(nil); //initializarea COM; it's required for every thread
   FADODb := TADODatabase.Create(nil);
   FADODb.DatabaseConnect := 'Provider=SQLOLEDB.1;Persist Security Info=False;User ID=sa;Initial Catalog=pubs;Data Source=CAM42';
   FADOQry := TADOQuery.Create(nil);
   FADOQry.Database := FADODb;
   FADOQry.ParamCheck := False;
   Fmt := TkbmMemTable.Create(nil);
end;

procedure TSQLThread.AfterExecute;
begin
   if Assigned(FADOQry) then
      begin
         FADOQry.Active:=False;
         FreeAndNIL(FADOQry);
      end;
   if Assigned(FADODb) then
      begin
         FADODb.Connected:=False;
         FreeAndNil(FADODb);
      end;
   if Assigned(Fmt) then
      begin
         FreeAndNil(Fmt);
      end;
   CoUnInitialize;
end;

procedure TForm1.FormCreate(Sender: TObject);
begin
   TCPServer.ThreadClass:=TSQLThread; //assign the TSQLThread class
   TCPServer.Active := True;
end;

procedure TForm1.FormDestroy(Sender: TObject);
begin
   TCPServer.Active := False;
end;

procedure TForm1.TCPServerExecute(AThread: TIdPeerThread);
var
   stream: TMemoryStream;
begin
   with AThread.Connection do
      begin
         stream := TMemoryStream.Create;
         try
            try
               with TSQLThread(AThread) do
                  begin
                     FADOQry.SQL.Text := ReadLn; //reads the sql command from socket
                     FADODb.Connected := True;
                     FADOQry.Open;
                     with Fmt do
                        begin
                           LoadFromDataSet(FADOQry, [mtcpoStructure,mtcpoOnlyActiveFields,mtcpoProperties,mtcpoLookup,mtcpoCalculated]);  //loads records returned by DB into memory table
                           Active := True;
                           SaveToStream(stream, [mtfSaveData, mtfSaveCalculated,mtfSaveLookup,mtfSaveNonVisible,mtfSaveBlobs,mtfSaveDef,mtfSaveIndexDef,mtfSkipRest,mtfSaveFiltered,mtfSaveInLocalFormat]); //saves the records in to a stream which can be optionally zipped
                        end;
                     FADOQry.Close;
                     FADODb.Connected := False;
                  end;
               WriteStream(stream); //writes the data to CL
               Disconnect;
               with TSQLThread(AThread).Fmt do
                  begin
                     EmptyTable;
                     Active := False;
                  end;
            except
               //if any errors
            end;
         finally
            stream.Free;
         end;
      end;
end;

end.
---------------------------------------------------------------------------

CL:

---------------------------------------------------------------------------
unit client;

interface

uses
  Windows, Messages, SysUtils, Classes, Graphics, Controls, Forms, Dialogs,
  IdBaseComponent, IdComponent, IdTCPConnection, IdTCPClient, StdCtrls,
  Grids, DBGrids, Db, kbmMemTable;

type
  TForm1 = class(TForm)
    tblc: TkbmMemTable;
    DataSource1: TDataSource;
    DBGrid1: TDBGrid;
    Button1: TButton;
    tcp: TIdTCPClient;
    procedure Button1Click(Sender: TObject);
  private
    { Private declarations }
  public
    { Public declarations }
  end;

var
  Form1: TForm1;

implementation

{$R *.DFM}

procedure TForm1.Button1Click(Sender: TObject);
var
   stream: TMemoryStream;
begin
   stream:=TMemoryStream.Create;
   try
      tcp.Connect;
      tcp.WriteLn('select * from authors');
      tcp.ReadStream(stream,-1,true);
      stream.Position:=0;
      tblc.LoadFromStream(stream);
      tcp.Disconnect;
   finally
      stream.Free;
   end;  
end;

end.
-----------------------------------------------------------------------------
The advantages: very high performance; scalability; can be used on multiple computers(like a cluster); instant deployment of client software;....

This is very easy model to resolve very complicated things. In the future (max 2 months) I'll make a set of componenets (under open source) for easy 3-tier development. In adition to those comps I'll make a ISAPI dll to make this arhitecture ready for Internet (this ISAPI dll is a trick very large used to pass firewalls).
These examples are made in 15 min. All I want to know if the community wants such components?
Thank you.





Please rate this article!
Skill level:
BeginnerExpert

Useful:
No!Very!

Overall rating:
PoorExcellent



Comments to this article
Write a new comment
Indy
    MIHAI ATANASIU (Aug 22 2002 4:23PM)

foarte interesanta treaba cu Indy.
O sa incerc si eu.
Respond

kbmMW
    Keith Blows (May 2 2002 2:20PM)

Very nice idea. The author of kbmMemTable (used  by your example) has developed a middleware library called kbmMW which does database thread-pooling and whole lot more.

Offers components for just about every database system. Check it out at http://www.components4developers.com.

At $149 once-off fee, this is a steal!
Respond

RE: kbmMW
anonymus (May 2 2002 10:33PM)

I know about kbmMW. Very good work, but:
1. I don't want a permanent connection between middleware server and db server(If I can't connect..send an exception to client);
2. I don't want middleware for every database: for small/medium projects I use Interbase with IBO; who really needs middleware is SQL Server and Oracle(my case SQL Server for medium/large);
3. I can control the lifes of connections thru db server(thanks to connection pooling and connection string);
4. no ISAPI dll for easy firewall pass.

Don't get wrong, I recommend kbmMW, but I like my ideea too.
Respond

RE: RE: kbmMW
Keith Blows (May 3 2002 8:47AM)

Simplicity is always a great goal. I think your idea is elegant and efficient (great tutorial on how to use Indy's thread object too!). However, you should also be aware that using SQL Server in this manner may violate your license. Many versions of SQL Server prohibit the use of middleware to gain access to the SQL data! One of Microsoft's many restrictive EULA practices.

Just wanted your readers to be aware that there is a more advanced option (with all the ncessary overhead) for abstracted database access. Abstracting a database to key business rules is absolutely key with bigger projects, as all the intelligence is built into the server objects and upgrading and improving the system is made that much simpler.

Regards

Keith
Respond

RE: RE: RE: kbmMW
aka aka (May 6 2002 9:54PM)

You must pay licenses by number of devices who query Sql Server with/without middleware.

Yes you are write: "bigger projects", like I said.
Respond

Simple....
    Hans Pieters (May 2 2002 12:23PM)

Interesting, go on ....please
Respond

RE: Simple....
yongzhen chen (May 30 2007 4:21AM)

LoadFromDataSet is very slooooooooooow when recordset is big than
60000....
or many user login, i am in trouble for this......

qry.cursortype = crforwardonly;
qry.cursorlocation = clclient

Respond

RE: RE: Simple....
saysansay say (Jul 31 2008 3:43AM)

hi Chen..i think this problems same with me, but i can solve this after study performance DB, so you must know how to maintance DB for server. You load data 60000?for what ?i think you can simple load data maybe you can use TOP in MSSQL and LIMIT and MySQL. SELECT TOP 100 * FROM TABLE, this can solve you problems.if you wan to fast load data.if you need information from other data and not in list data, you can create method search or find data.this just my opinion chen...:D
Respond














 
Sign up to consume product discounts for Bronze memberships !

read more


   


  Community Ad of
I. Siticov
 
   














 







     
  Copyright © 2000 - 2007 delphi3000.com - All rights reserved. Terms of use. || Privacy
delphi3000.com is a service by bluestep.com IT-Services GmbH (Vienna)