How to create a custom DataSet

Create you own Custom DataSet!
by William Anthony

Have you ever needed/wanted to create your own personal Database, well this is the place for you!
[HybridSPOT] is proud to present a multi-part series on just that, starting with the TDataSet Component
we will create a new and powerful OPEN-SOURCE Database Engine.

This tutorial will teach you about the basics of Developing a TDataSet,
and trust me it isn't such a simple task. -_-

the TDataSet Structure
The Naked Truth


TNAKEDDataSet = class(TDataSet)
protected
function AllocRecordBuffer: PChar; override;
function GetBookmarkFlag(Buffer: PChar): TBookmarkFlag; override;
function GetCanModify: Boolean; override;
function GetRecord(Buffer: PChar; GetMode: TGetMode; DoCheck: Boolean): TGetResult; override;
function IsCursorOpen: Boolean; override;

{ THESE THREE WE CAN LEAVE OUT }
function GetRecNo: Integer; override;
function GetRecordCount: Integer; override;
function GetRecordSize: Word; override;

procedure FreeRecordBuffer(var Buffer: PChar); override;
procedure GetBookmarkData(Buffer: PChar; Data: Pointer); override;
procedure InternalAddRecord(Buffer: Pointer; Append: Boolean); override;
procedure InternalClose; override;
procedure InternalFirst; override;
procedure InternalGotoBookmark(Bookmark: Pointer); override;
procedure InternalHandleException; override;
procedure InternalInitFieldDefs; override;
procedure InternalInitRecord(Buffer: PChar); override;
procedure InternalLast; override;
procedure InternalOpen; override;
procedure InternalPost; override;
procedure InternalSetToRecord(Buffer: PChar); override;
procedure SetBookmarkData(Buffer: PChar; Data: Pointer); override;
procedure SetBookmarkFlag(Buffer: PChar; Value: TBookmarkFlag); override;
procedure SetFieldData(Field: TField; Buffer: Pointer; NativeFormat: Boolean); override;

public
property BufferCount;
property Buffers;
property Fields;
property FieldValues;

function GetFieldData(Field: TField; Buffer: Pointer; NativeFormat: Boolean): Boolean; override;
published
property Active;
property AfterCancel;
property AfterClose;
property AfterDelete;
property AfterEdit;
property AfterInsert;
property AfterOpen;
property AfterPost;
property AfterRefresh;
property AfterScroll;
property BeforeCancel;
property BeforeClose;
property BeforeDelete;
property BeforeEdit;
property BeforeInsert;
property BeforeOpen;
property BeforePost;
property BeforeRefresh;
property BeforeScroll;
property CanModify;
property FieldDefs;
property Filter;
property FilterOptions;
property ObjectView default False;
property OnCalcFields;
property OnDeleteError;
property OnEditError;
property OnNewRecord;
property OnPostError;
property RecNo;
property RecordCount;
property RecordSize;
property State;
end;

Well it looks a bit BULKY but that's what it needs in the REAL WORLD

the Abstract Functions
In earlier versions of Delphi this class had LOTS of abstract procedures, now it contains but a few:

InternalOpen
Type Abstract
Use Generate Fields
Bind Fields
Open the Source of Data (File - Memory - Other)

InternalClose
Type Abstract
Use Un-Bind Fields
Close the Source of Data (File - Memory - Other)

IsCursorOpen
Type Abstract Function  
Use Return DataSet Active Status (is dataset open)

InternalHandleException
Type Abstract
Use Respond to any exception that is generated

GetRecord
Type Abstract Function  
Use Get record data,
There record position has to be updated within the procedure VIA the GETMODE param. the resulting value must be updated aswell, for example if the file goes off the EOF or BOF.


the BookMark System
You might be asking yourself as i once did: "WELL I DONT NEED BOOKMARKS FOR THIS SIMPLE DATASET!"


Well guess what YOU DO.It seems that the Bookmark is not really a Bookmark more of a ID for each one of the Temporary Buffer. in standard it keeps about 3 records in memory

Prior - Current - Next.
Each containing there WHERE and WHO:

WHERE is the BookmarkFlag (Prior, Current, Next) for orientation
WHO is the BookmarkData (Record No) for ID and Placement.

So in turn we must make a record for this data, beeing that it is fixed width we can place it infront of the main data, but in the case of this sample i fused them both together.

the BookMark procedures
SetBookmarkData
Type Procedure
Use Set RecordNo to Active Buffer
No thinker ^_^

SetBookmarkFlag
Type Procedure
Use Set BookmarkFlag to Active Buffer
No thinker ^_^ aswell

GetBookmarkData
Type Procedure
Use Get RecordNo from Active Buffer
No thinker ^_^ x 3

GetBookmarkFlag
Type Function
Use Get BookmarkFlag from Active Buffer
No thinker ^_^ exp 4

InternalSetToRecord
Type Procedure
Use Get RecordNo from Active Buffer and set it as current record
No thinker o_O

InternalGotoBookmark
Type Procedure
Use Get RecordNo from DATA and set it as current record
No thinker O_o EASY AS "3.1415..." (Pi)


the Navegation  
Now that we see that the last part isn't that hard we move to the navegation,
if you have used a DataSet before this shuld be easy.

InternalFirst
Type Procedure
Use We set our Current Position to -1 (in some cases) BOF
No thinker ^_^

InternalLast
Type Procedure
Use We set our Current Position to RecordCount (in some cases) EOF
No thinker ^_^



the BUFFERS of Terror
Now we come to the next level of the DataSet "the Buffer System"


AllocRecordBuffer
Type Function
Use Return the new memory allocation for the buffer
result := getMemory(MYRECORDSIZEWITHBOOKMARKINFO);
No thinker ^_^

FreeRecordBuffer
Type Procedure
Use HERE IS THE CODE "FreeMem(Buffer);"
No thinker ^_^ x 1000



Well that wasnt that bad was it?
the Data Processing  

Ahh yes we are getting more work to do!

InternalPost
Type Procedure
Use Post changes if any,
We must check if the State is Edit or Insert, since this will have an effect on our process, if the state is set to Edit this meens that we must write our changes on the current record no questions asked, else we must go to the end and add a new record, then we write.
NOT EVEN A MICRO-Thinker o_O

InternalAddRecord
Type Procedure
Use Add current buffered record to file, same as the last but this time we just add it no questions asked.

NOT EVEN A MICRO-Thinker O_O x 2


the Field System
Yes the work is yet to come!

SetFieldData
Type Procedure
Use Convert and Pass the data from the Active Buffer to the Field Buffer
the param contains the FieldClass and the Field Buffer.
the active buffer is given by the ActiveBuffer function.
Semi-Thinker ^_^ depends on the Table Format

GetFieldData
Type Function
Use Convert and Pass the data from the Field Buffer to the Active Buffer
the param contains the FieldClass and the Field Buffer.
the active buffer is given by the ActiveBuffer function.

Result Value is boolean, it the data was passed OK then it returns TRUE

Semi-Thinker ^_^ depends on the Table Format again


the Usefull yet Not Needed
I still use them do, even if they to make a mess if not used properly


GetRecNo
Type Function
Use Return the current Record No

GetRecordCount
Type Function
Use Return the Record Count

GetRecordSize
Type Function
Use Return the Record Size w/o Bookmark Info

SetRecNo
Type Procedure
Use Set the current Record No



These procedures are used by some controls like TDBGrid and such.
the Holy-Grail
It is time to learn!

All these procedures have been detailed for you to figure out, the code is WAY SIMPLE and WELL COMMENTATED so you shuldnt be lost for long.

This is a sample of a TINY User Database complete with:

User Name 20 Characters
Password 10 Characters
AccessLevel 4 Bytes [Integer]

now in two days we will learn to make a DBF file reader! yes you wont have to use that bulky DBE anymore! and you will learn how different file formats work O_o

Download NOW



I hope this tutorial (instruction manual) has gotten some of you to try and build your skill up,
remember that some if not all of knowledge comes from practice, so... repeat and repeat, make your own DataSet's until you get it working just fine!

-William Anthony
HybridGRAF Interactive Solutions