question

HasmukhGinoya-8695 avatar image
0 Votes"
HasmukhGinoya-8695 asked HasmukhGinoya-8695 edited

ADODB Record set- How to fetch Record Asynchronously in between without loading all records into heap memory

I am using ADODB C++ for Execute Query like Select * from table. This table having the large data around 4GB. So when i Execute Query then it is fetching all the records and increase heap memory and load all the records on the client machine virtaul memory.
this is my code

// Create Record set Object
ADODB::RecordsetPtr m_pRecordSetData= NULL;
HRESULT hr = m_pRecordSetData.CreateInstance(
_uuidof(ADODB::Recordset));
m_pRecordSetData->CursorType = ADODB::adOpenStatic;
m_pRecordSetData->CursorLocation = ADODB::adUseClient;
m_pRecordSetData->LockType = ADODB::adLockReadOnly;

// Create Command Object
ADODB::CommandPtr pCmdPtr = NULL;
HRESULT hr = pCmdPtr.CreateInstance(
_uuidof(ADODB::Command));
if (FAILED(hr))
{
return pCmdPtr;
}
pCmdPtr->PutCommandType(ADODB::adCmdText);
pCmdPtr->PutCommandTimeout(0);
pCmdPtr->ActiveConnection = m_pConnection;
pCmdPtr->CommandText = L"Select from Table";
// Execute Query Select from table*
_variant_t vtConn;
vtConn.vt = VT_ERROR;
vtConn.scode = DISP_E_PARAMNOTFOUND;
HRESULT hr = m_pRecordSetData->Open((_variant_t((IDispatch
)pCmdPtr)), vtConn,
ADODB::adOpenStatic, ADODB::adLockReadOnly, -1);


So When We execute last line Recordset Open this will block until all the record is fetched. So it consume all the heap memory of the system.

Expectation / Requirement : Can we fetch the record batch wise / few rows only? , So we can allocated that heap memory and release it. So this will not consume all client side heap memory. Can we fetch asynchronously we will get the next record in batch wise? So we don't want to fetch all record in one go.

windows-apic++
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.


Check if it works faster and with less memory consumption if you replace adOpenStatic (in several places) with adOpenDynamic, adOpenForwardOnly or adOpenKeyset. Maybe also replace adUseClient with asUseServer.


0 Votes 0 ·

1 Answer

XiaopoYang-MSFT avatar image
0 Votes"
XiaopoYang-MSFT answered HasmukhGinoya-8695 edited
· 5
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Thanks for Reply
This both property will not work.
Our problem is when we run Recordset open command then can we prevent to all the recordset into the client memory?

m_pRecordSetData->Open((_variant_t((IDispatch )pCmdPtr)), vtConn,
ADODB::adOpenStatic, ADODB::adLockReadOnly, -1);
This line is loading all the record on virtual memory of the client. So can we make then it will load only few record and we use it and then we request for next few like that?

0 Votes 0 ·

Can This SO Question Limiting memory-usage of ADO recordset help you?


0 Votes 0 ·

Yes i want to do the same way , but that case they have fixed query with cardnumber .
While in my case we have query only select * from table only and it will fetch all recordset data into heap memory.

0 Votes 0 ·

Can You limit the m_pRecordSetData MaxRecords Property?

0 Votes 0 ·

I have Set MaxRecords like
m_pRecordSetData ->PutMaxRecords(100);
Although it is fetching whole my Recordset. I have checked that also.

you can please suggest me if i am doing any wrong configuration.

0 Votes 0 ·