Download File : Power Automate and JavaScript

What is Issue?

When working on D365 CRM, Some time user wants to download file from SharePoint, OneDrive or CRM Notes itself. So how you will do it in shortest way?

The Reason-

Downloading file from SharePoint or any other cloud location it will be time consuming process if you choose Web service or any WCF service.

Solution-

Now Power Automate provides less code and quick solutions. Power Automate has all the connectors and Actions which help you to connect these cloud locations and process on the files.

Let’s see a scenario, I want to download a file on button click on my CRM Account form. The file is located in my account record Timeline Notes.

Let’s go step by step.

First of all, as this is action on click of button on form, we can choose “When Http Request Received” trigger in Power Automate. And in HTTP request we need record id i.e. accountid to retrieve specific Note attachment i.e. “CustomerInfoDoc”-

Now from above step we will get the Document – File name and Document body (base64) content.

So we need a variable to store files in array and another variable for document content-

Setting the variables from the Notes list which we got from “List rows” action-

We need to append array with below json-

{
“Name”: @{items(‘Apply_to_each’)?[‘filename’]},
“ContentBytes”: @{items(‘Apply_to_each’)?[‘documentbody’]}
}

We will use Name and ContentBytes in JavaScript Code which will be in the response of HttpRequest.

Now once file we got file content we need to send them in response-

We need the only file from Notes, In List Row action we can add row count =1, but still in response we can set formula in body first(variable(‘ContentArray’). As this is document in response add headers “Content-type”:”multipart/form-data”

Now our complete flow is ready and it looks like –

So now remaining part is calling the flow from button to download the file-

Add the below javascript and call “callPowerAutomateFlow” function on button click. Below is complete code for download –

function callPowerAutomateFlow(formContext) { 
    id = formContext.data.entity.getId().replace("{", "").replace("}", ""); 
    var account = '{ "accountid": "' + id + '"'+ '}'; 
    var flowUrl = "https://prod-04.centralindia.logic.azure.com:443/workflows/xxxxxxxxxxxxxxxxxxxxxxxxxx/triggers/manual/paths/invoke?api-version=2016-06-01&sp=%2Ftriggers%2Fmanual%2Frun&sv=1.0&sig=xxxxxxxxxxxx"; 
 
    var req = new XMLHttpRequest(); 
    req.open("POST", flowUrl, true); 
    req.setRequestHeader("Accept", "application/json"); 
    req.setRequestHeader("Content-Type", "application/json; charset=utf-8"); 
    req.onreadystatechange = function () { 
        if (this.readyState === 4) { 
            req.onreadystatechange = null; 
            if (this.status === 200) { 
                    var result = JSON.parse(this.response); 
                    var fileName = result.Name; 
                    var fileCont = result.ContentBytes; 
                    var filebyte = base64ToBufferArray(fileCont); 
                    downloadFile(fileName, [filebyte]); 
            } else { 
                Xrm.Utility.alertDialog(this.responseText); 
            } 
        } 
    }; 
    req.send(account); 
} 
 
function base64ToBufferArray(base64content) { 
    var binaryString = window.atob(base64content); 
    var binaryLen = binaryString.length; 
    var bytes = new Uint8Array(binaryLen); 
    for (var i = 0; i < binaryLen; i++) { 
        var ascii = binaryString.charCodeAt(i); 
        bytes[i] = ascii; 
    } 
    return bytes; 
} 
 
function downloadFile(name,data) { 
        var blob = new Blob(data, { type: "octet/stream" }), 
            url = window.URL.createObjectURL(blob); 
        if (window.navigator && window.navigator.msSaveOrOpenBlob) { 
            window.navigator.msSaveOrOpenBlob(blob, name); 
        } else { 
            const url = window.URL.createObjectURL(blob); 
            const a = document.createElement('a'); 
            document.body.appendChild(a); 
            a.href = url; 
            a.download = name; 
            a.click(); 
            window.URL.revokeObjectURL(url); 
            document.body.removeChild(a); 
        } 
}

That’s it!! when I am clicking on Download button in CRM Account record, it is downloading my Attachment file from Notes which is with title “CustomerInfoDoc”.

Hope this will help…
Enjoy Ms CRM!!!

Follow on Facebook- FB: MSCRM16Tech

Attaching File from SharePoint Location in Email using Power Automate while working on D365 CRM

What is Issue?

When working on D365 CRM, Some processes needs to send notification as well as approval mails to particular users. In these emails some time users want the files to be attached which is uploaded for any particular record in CRM. Developer faces issue to get the file from SharePoint, because of very few columns available in SharePoint which is created by D365 CRM to SharePoint Integration. So the issues are like how to get the file from SharePoint and attach it email?

The Reason-

Using Workflow or custom plugin it will be much time consuming and also complex coding required with this approach. Developer can choose this approach for sending out simple notifications, emails. But when the requirements comes to file attachment from SharePoint or any other cloud location it will be difficult to handle with this approach. Also if the file is in SharePoint location which is configured using Document management in CRM, location will have limited set of columns, so it becomes difficult to query the SharePoint site using API also.

Solution-

For the first issue, which approach to choose in these scenarios-
Microsoft is already recommending users to use Power Automat instead of CRM Workflows. Power Automate is having multiple connectors and actions available which reduces the coding as well as time efforts. So we can choose Power Automate.

Now second Issue, How to get file from SharePoint location which is configured using Document Management in CRM?
As I already mentioned, Power Automate already having hundreds of connectors and actions, we can use the Get File Action in Power Automate. Now again you will be worried about how you can get particular file from this location as Tittle or any other field will not help here. So we can use FileLeafRef column filter for that.

Lets take scenario –

When user have opportunity where he uploads the template for Opportunity Activity logs in Document section. But before closing opportunity user needs the log template approved by his manager. Suppose there is flag when user submits the approval request.

  1. Trigger action for my Power Automate flow will be “Microsoft Dataverse when a row is added, modified or deleted where I will set row filter as AwaitngApprovalflag eq 1
  2. Now we need to initialize the Array which will work as byte array to store the file retrieved from the SharePoint.
  3. Add new SharePoint action “Get File(Properties Only)” and set SharePoint details with Filter Query – FileLeafRef eq “filename.xlsx” (this will be my log file template name)

4. Now add the SharePoint action “Get File Content” and set site address as SharePoint site, File Identifier as identifier from last action i.e. “Get File (Properties Only)”. This will automatically add Apply to Each loop.
5. Append the Array variable with File Content and file name from the above action “Get File Content”.
Example-
{
“FileName”:”fileName.xlsx”,
“ContentByte”:FileContent
}

7. Add action “Send Email With Option” and set participant and email body, and other details. In attachment section set the Array Variable which we initialized and configured with file content.

8. Add condition action and check the response from the manager in Email like he clicked on Approve or Reject button in email. and based on condition perform the next actions on your D365 CRM Record.

9. The Complete Flow looks like-

Hope this will help…
Enjoy Ms CRM!!!

Follow on Facebook- FB: MSCRM16Tech

Batch operation ($batch) using OData API Dynamics 365

What is Issue?

There are OData API provided by Microsoft to perform operations on D365 Tables like Create, update, delete and retrieve. These operations can be perform individually by building the OData API using some tools or with your knowledge. But some time there are the cases where you need to call the bulk create, bulk update, bulk delete or some bulk create and bulk update/delete simultaneously. If there are more than one records, you need to hit OData API one by one multiple times which takes cost in terms of time and performance of the application which is calling OData API. Is there a way to combine these all requests in one call and execute?

The Reason-

There is way to perform the CRUD operations on D365 Tables using OData, but everyone aware of the single operations. Microsoft provides the way to perform Batch operation on Tables using OData API. Lets see in solution for this.

Solution-

Microsoft D365 provides the OData for batch operation to perform using “$batch” like-

https://your_org.crm8.dynamics.com/api/data/v8.2/$batch

You need to include the all other CRUD (POST, PATCH, DELETE, GET) OData APIs in the body for the above request URL.

This request works like all other OData APIs using OAuth2.0 authentication with proper headers added.

Lets take .Net application where I need to perform the Create, update and some delete operation in one call.

You have to prepare your request as below-

Preparing the Batch Call-

#region prepare Batch Call
        private string BatchQueryCall(string crmRestQuery, string batchName, string requestBody)
        {
            string funResponse = null;
            try
            {
                HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post, crmRestQuery);
                //add header parameters
                request.Headers.Add("Authorization", "Bearer " + oauthToken);
                request.Headers.Add("Accept", "application/json");
                request.Content = new StringContent(requestBody);
                request.Content.Headers.ContentType = MediaTypeHeaderValue.Parse("multipart/mixed;boundary=" + batchName);

                HttpResponseMessage response = httpClient.SendAsync(request).Result;
                string responseString = response.Content.ReadAsStringAsync().Result;

                if (response.IsSuccessStatusCode)
                {
                    funResponse = responseString;
                }
                else
                    funResponse = responseString;

            }
            catch (Exception ex)
            {
                funResponse = ex.Message;
            }
            return funResponse;
        }
        #endregion

Preparing to Call Batch Operation-

Suppose I have to create one record, update one record and delete 2 records in same call.
Below code helps you to understand preparing the Batch call-

  #region bulk Operations
        public string BulkOperationsAPI(string entityname)
        {
            #region Local Variable
            string crmRestQuery = apiUrl + "$batch";
            string batchName = "batch_bt123";
            string changeSetVar = "changeset_ch123";
            int coRelId = 1;
            string changeSet1 = null;
            string requestBody = "--" + batchName + Environment.NewLine
                             + "Content-Type:multipart/mixed;boundary=" + changeSetVar + Environment.NewLine + Environment.NewLine + Environment.NewLine;
            #endregion

            #region Create
                //Creating one record
                CreateUpdateOpportunity entity = new CreateUpdateOpportunity();

                entity.transactioncurrencyid_odata_bind = "/transactioncurrencies(c63ba581-6bc6-e811-a96f-000d3af04fb0)";
                entity.estimatedvalue = 1000;
                entity.estimatedclosedate = "2021-01-01";
                entity.name = "Generated From C# Code";
                entity.parentaccountid_odata_bind = "/accounts(3b3b7c71-61d2-ea11-a813-000d3af0205e)";

                changeSet1 = PrepareReuqestBody("POST", changeSetVar, coRelId++, entityname, null, entity);

                if (!string.IsNullOrEmpty(changeSet1))
                    requestBody += changeSet1 + Environment.NewLine;
            
            #endregion

            #region Update
                //updating One record
                CreateUpdateOpportunity entity1 = new CreateUpdateOpportunity();

                entity1.transactioncurrencyid_odata_bind = "/transactioncurrencies(c63ba581-6bc6-e811-a96f-000d3af04fb0)";
                entity1.estimatedvalue = 1000;
                entity1.estimatedclosedate = "2021-01-01";
                entity1.name = "Generated From CSharp Code";
                entity1.parentaccountid_odata_bind = "/accounts(3b3b7c71-61d2-ea11-a813-000d3af0205e)";

                changeSet1 = PrepareReuqestBody("PATCH", changeSetVar, coRelId++, entityname, "20e6bb72-2861-eb11-a812-0022486e93ce", entity1);

                if (!string.IsNullOrEmpty(changeSet1))
                    requestBody += changeSet1 + Environment.NewLine;
            #endregion

            #region Delete
            //delete 1st record  
            changeSet1 = PrepareReuqestBody("DELETE", changeSetVar, coRelId++, entityname, "29e6bb72-2861-eb11-a812-0022486e93ce", null);
                if (!string.IsNullOrEmpty(changeSet1))
                    requestBody += changeSet1 + Environment.NewLine;

            //delete 2nd record
            changeSet1 = PrepareReuqestBody("DELETE", changeSetVar, coRelId++, entityname, "27e6bb72-2861-eb11-a812-0022486e93ce", null);
            if (!string.IsNullOrEmpty(changeSet1))
                requestBody += changeSet1 + Environment.NewLine;
            #endregion


            requestBody += "--" + changeSetVar + "--" + Environment.NewLine;
            requestBody += "--" + batchName + "--";
            
            //format the body URLs
            requestBody = requestBody.Replace("\\/", "/");  

            //call the batch Operations
            return BatchQueryCall(crmRestQuery, batchName, requestBody);
        }
        #endregion

Preparing the body content of Batch call-

In above code there is call made to the function PrepareRequestBody-
This function will prepare the body content of Batch call.

        #region Prepare Request Body
        private string PrepareReuqestBody(string method, string changeset, int contentId, string entityName, string recordid, object entity)
        {

            string x = null;
            switch (method.ToUpper())
            {
                case "PATCH":
                    using (MemoryStream streamOpportunitySerialize = new MemoryStream())
                    {
                        DataContractJsonSerializer ser = new DataContractJsonSerializer(entity.GetType());
                        ser.WriteObject(streamOpportunitySerialize, entity);
                        streamOpportunitySerialize.Position = 0;
                        StreamReader srOpportunity = new StreamReader(streamOpportunitySerialize);
                        string objectJSON = srOpportunity.ReadToEnd();
                        objectJSON = objectJSON.Replace("_odata_bind", "@odata.bind");

                        x = "--" + changeset + Environment.NewLine
                            + "Content-Type: application/http" + Environment.NewLine
                            + "Content-Transfer-Encoding: binary" + Environment.NewLine
                            + "Content-ID: " + contentId + Environment.NewLine + Environment.NewLine
                            + "PATCH " + apiUrl + entityName + "(" + recordid + ") HTTP/1.1" + Environment.NewLine
                            + "Content-Type: application/json;type=entry" + Environment.NewLine
                            + "Accept:application/json" + Environment.NewLine + Environment.NewLine
                            + objectJSON + Environment.NewLine + Environment.NewLine;
                    }
                    break;
                case "POST":
                    using (MemoryStream streamOpportunitySerialize = new MemoryStream())
                    {
                        DataContractJsonSerializer ser = new DataContractJsonSerializer(entity.GetType());
                        ser.WriteObject(streamOpportunitySerialize, entity);
                        streamOpportunitySerialize.Position = 0;
                        StreamReader srOpportunity = new StreamReader(streamOpportunitySerialize);
                        string objectJSON = srOpportunity.ReadToEnd();
                        objectJSON = objectJSON.Replace("_odata_bind", "@odata.bind");

                        x = "--" + changeset + Environment.NewLine
                            + "Content-Type: application/http" + Environment.NewLine
                            + "Content-Transfer-Encoding: binary" + Environment.NewLine
                            + "Content-ID: " + contentId + Environment.NewLine + Environment.NewLine
                            + "POST " + apiUrl + entityName + " HTTP/1.1" + Environment.NewLine
                            + "Content-Type: application/json;type=entry" + Environment.NewLine
                            + "Accept:application/json" + Environment.NewLine + Environment.NewLine
                            + objectJSON + Environment.NewLine + Environment.NewLine;
                    }

                    break;
                case "DELETE":
                    x = "--" + changeset + Environment.NewLine
                    + "Content-Type: application/http" + Environment.NewLine
                    + "Content-Transfer-Encoding: binary" + Environment.NewLine
                    + "Content-ID: " + contentId + Environment.NewLine + Environment.NewLine
                    + "DELETE " + apiUrl + entityName + "(" + recordid + ") HTTP/1.1" + Environment.NewLine
                    + "Accept:application/json" + Environment.NewLine + Environment.NewLine;
                    break;
            }

            return x;
        }
        #endregion

Request Body content will look like-

--batch_bt123
Content-Type:multipart/mixed;boundary=changeset_ch123


--changeset_ch123
Content-Type: application/http
Content-Transfer-Encoding: binary
Content-ID: 1

POST https://your_org.crm8.dynamics.com/api/data/v8.2/opportunities HTTP/1.1
Content-Type: application/json;type=entry
Accept:application/json

{"estimatedclosedate":"2021-01-01","estimatedvalue":1000,"name":"Generated From C# Code","parentaccountid@odata.bind":"/accounts(3b3b7c71-61d2-ea11-a813-000d3af0205e)","transactioncurrencyid@odata.bind":"/transactioncurrencies(c63ba581-6bc6-e811-a96f-000d3af04fb0)"}


--changeset_ch123
Content-Type: application/http
Content-Transfer-Encoding: binary
Content-ID: 2

PATCH https://your_org.crm8.dynamics.com/api/data/v8.2/opportunities(d659b4d6-0f5b-eb11-a812-6045bd727e64) HTTP/1.1
Content-Type: application/json;type=entry
Accept:application/json

{"estimatedclosedate":"2021-01-01","estimatedvalue":1000,"name":"Generated From CSharp Code","parentaccountid@odata.bind":"/accounts(3b3b7c71-61d2-ea11-a813-000d3af0205e)","transactioncurrencyid@odata.bind":"/transactioncurrencies(c63ba581-6bc6-e811-a96f-000d3af04fb0)"}


--changeset_ch123
Content-Type: application/http
Content-Transfer-Encoding: binary
Content-ID: 3

DELETE https://your_org.crm8.dynamics.com/api/data/v8.2/opportunities(46c9eab1-5ac2-4673-a55a-51cb886db902) HTTP/1.1
Accept:application/json


--changeset_ch123
Content-Type: application/http
Content-Transfer-Encoding: binary
Content-ID: 4

DELETE https://your_org.crm8.dynamics.com/api/data/v8.2/opportunities(d659b4d6-0f5b-eb11-a812-6045bd727e64) HTTP/1.1
Accept:application/json


--changeset_ch123--
--batch_bt123--

Response recieved will be like-

  --batchresponse_319a4c10-b7fe-4bf0-b4a6-da016d1c0319
Content-Type: multipart/mixed; boundary=changesetresponse_ee30dcdb-1094-4c24-8170-262eae9336a4

--changesetresponse_ee30dcdb-1094-4c24-8170-262eae9336a4
Content-Type: application/http
Content-Transfer-Encoding: binary
Content-ID: 1

HTTP/1.1 204 No Content
OData-Version: 4.0
Location: https://your_org.crm8.dynamics.com/api/data/v8.2/opportunities(3d4916b4-3861-eb11-a812-0022486e93ce)
OData-EntityId: https://your_org.crm8.dynamics.com/api/data/v8.2/opportunities(3d4916b4-3861-eb11-a812-0022486e93ce)


--changesetresponse_ee30dcdb-1094-4c24-8170-262eae9336a4
Content-Type: application/http
Content-Transfer-Encoding: binary
Content-ID: 2

HTTP/1.1 204 No Content
OData-Version: 4.0
Location: https://your_org.crm8.dynamics.com/api/data/v8.2/opportunities(d659b4d6-0f5b-eb11-a812-6045bd727e64)
OData-EntityId: https://your_org.crm8.dynamics.com/api/data/v8.2/opportunities(d659b4d6-0f5b-eb11-a812-6045bd727e64)


--changesetresponse_ee30dcdb-1094-4c24-8170-262eae9336a4
Content-Type: application/http
Content-Transfer-Encoding: binary
Content-ID: 3

HTTP/1.1 204 No Content
OData-Version: 4.0


--changesetresponse_ee30dcdb-1094-4c24-8170-262eae9336a4
Content-Type: application/http
Content-Transfer-Encoding: binary
Content-ID: 4

HTTP/1.1 204 No Content
OData-Version: 4.0


--changesetresponse_ee30dcdb-1094-4c24-8170-262eae9336a4--
--batchresponse_319a4c10-b7fe-4bf0-b4a6-da016d1c0319--

For Token Generation you can refer the code in my last blog – https://mscrm16tech.com/2021/01/21/crud-operations-on-dynamics-365-crm-tables-with-odata-api/

Hope this will help…
Enjoy Ms CRM!!!

Follow on Facebook- FB: MSCRM16Tech

CRUD Operations on Dynamics 365 CRM Tables with ODATA API

What is Issue?

Performing CRUD operation on Dynamics 365 CRM using API (Rest/ODATA) from other applications.

The Reason-

While performing integration of other application with CRM, creating custom Web API and webservice is having much efforts in terms of time and development. Microsoft already made developers job easy by providing the ODATA API available for their products. How to consume that API in other applications.

Solution-

While performing integration of other application with CRM, rather than creating the custom API or webservice, we can utilize ODATA APIs that can be consumed with the OAuth 2.0 authentication.

Lets take .net application which is consuming ODATA API.

Some time we don’t want to include any third party dll due to some environmental restrictions. so the sample code added below is using only system libraries.

(In case of Json serialization, you can utilize Newtonsoft.Json to avoid the classes to be written manually)

Create records using ODATA API-

#region Create API Call
        /// <summary>
        /// Create the entity Record using API Call
        /// </summary>
        /// <param name="entityname">entity Name</param>
        /// <param name="entity">entity Object with parameter values</param>
        /// <returns></returns>
        public string CreateAPI(string entityname, object entity)
        {
            string funResponse = null;
            string query = entityname;
            string crmRestQuery = apiUrl + query;

            try
            {
                using (MemoryStream streamOpportunitySerialize = new MemoryStream())
                {
                    DataContractJsonSerializer ser = new DataContractJsonSerializer(entity.GetType());
                    ser.WriteObject(streamOpportunitySerialize, entity);
                    streamOpportunitySerialize.Position = 0;
                    StreamReader srOpportunity = new StreamReader(streamOpportunitySerialize);
                    string objectJSON = srOpportunity.ReadToEnd();
                    objectJSON = objectJSON.Replace("_odata_bind", "@odata.bind");

                    HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post, crmRestQuery);
                    //add header parameters
                    request.Headers.Add("Authorization", "Bearer " + oauthToken);
                    request.Content = new StringContent(objectJSON);
                    request.Content.Headers.ContentType = MediaTypeHeaderValue.Parse("application/json; charset=utf-8");

                    HttpResponseMessage response = httpClient.SendAsync(request).Result;
                    string responseString = response.Content.ReadAsStringAsync().Result;

                    if (response.IsSuccessStatusCode)
                    {
                        string _recordUrl = response.Headers.GetValues("OData-EntityId").FirstOrDefault();
                        string[] splitRetrievedData = _recordUrl.Split('[', '(', ')', ']');
                        funResponse = splitRetrievedData[1];
                    }
                    else
                        funResponse = responseString;

                }
            }
            catch (Exception ex)
            {
                funResponse = ex.Message;
            }
            return funResponse;
        }
        #endregion

Update records using ODATA API-

  #region Update API Call
        /// <summary>
        /// Update the entity record using API Call
        /// </summary>
        /// <param name="entityname">entity name</param>
        /// <param name="entityid">entity id</param>
        /// <param name="entity">entity Object with parameter values</param>
        /// <returns></returns>
        public string UpdateAPI(string entityname, string entityid, object entity)
        {
            string funResponse = null;
            string query = entityname + "(" + entityid + ")";
            string crmRestQuery = apiUrl + query;
            try
            {
                using (MemoryStream streamOpportunitySerialize = new MemoryStream())
                {

                    DataContractJsonSerializer ser = new DataContractJsonSerializer(entity.GetType());
                    ser.WriteObject(streamOpportunitySerialize, entity);
                    streamOpportunitySerialize.Position = 0;
                    StreamReader srOpportunity = new StreamReader(streamOpportunitySerialize);
                    string objectJSON = srOpportunity.ReadToEnd();
                    objectJSON = objectJSON.Replace("_odata_bind", "@odata.bind");

                    HttpMethod method = new HttpMethod("PATCH");
                    HttpRequestMessage request = new HttpRequestMessage(method, crmRestQuery);
                    //add header parameters
                    request.Headers.Add("Authorization", "Bearer " + oauthToken);
                    request.Headers.Add("If-Match", "*");      //preventing creation of new record if no record found with ID
                    request.Content = new StringContent(objectJSON);
                    request.Content.Headers.ContentType = MediaTypeHeaderValue.Parse("application/json; charset=utf-8");

                    HttpResponseMessage response = httpClient.SendAsync(request).Result;
                    string responseString = response.Content.ReadAsStringAsync().Result;
                    if (response.IsSuccessStatusCode)
                    {
                        funResponse = "Sucessfully Updated Record!!";
                    }
                    else
                        funResponse = responseString;
                }
            }
            catch (Exception ex)
            {
                funResponse = ex.Message;
            }
            return funResponse;
        }
        #endregion

Delete records using ODATA API-

#region Delete API Call
        /// <summary>
        /// Delete the entity Record using API Call
        /// </summary>
        /// <param name="entityname">entity Name</param>
        /// <param name="entityid">Entity id</param>
        /// <returns></returns>
        public string DeleteAPI(string entityname, string entityid)
        {
            string funResponse = null;
            string query = entityname + "(" + entityid + ")";
            string crmRestQuery = apiUrl + query;

            try
            {
                HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Delete, crmRestQuery);
                //add header parameters
                request.Headers.Add("Authorization", "Bearer " + oauthToken);
                request.Headers.Add("Accept", "application/json; charset=utf-8");

                HttpResponseMessage response = httpClient.SendAsync(request).Result;
                string responseString = response.Content.ReadAsStringAsync().Result;
                if (response.IsSuccessStatusCode)
                    funResponse = "Deleted!!!";
                else
                    funResponse = responseString;
            }
            catch (Exception ex)
            {
                funResponse = ex.Message;
            }
            return funResponse;
        }
        #endregion

Retrieve Single record using ODATA API-

 /// <summary>
        /// Retrieve single using API Call
        /// </summary>
        /// <param name="entityPluralName">entity Name</param>
        /// <param name="entityId">entity id</param>
        /// <param name="entityObject">blank entity object</param>
        /// <param name="selectColumns">select columns to retrieve in API Call</param>
        /// <returns>Object of entity</returns>
        public object RetrieveSingle(string entityPluralName,string entityId,object entityObject, string selectColumns=null)
        {
            string query = "("+entityId+")";
            if (selectColumns != null) {
                query += "?$select=" + selectColumns;
            }
            string crmRestQuery = apiUrl + entityPluralName + query;
            try
            {
                    HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, crmRestQuery);
                    //add header parameters
                    request.Headers.Add("Prefer", "odata.include-annotations=\"*\"");//for formatted values
                    request.Headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/XML"));
                    request.Headers.Add("Authorization", "Bearer " + oauthToken);

                    //send request
                    HttpResponseMessage response = httpClient.SendAsync(request).Result;
                    string responseString = response.Content.ReadAsStringAsync().Result;
                    responseString = FormatResponse(responseString);

                    using (var ms = new MemoryStream(Encoding.Unicode.GetBytes(responseString)))
                    {
                        // Deserialization from JSON  
                        DataContractJsonSerializer deserializer = new DataContractJsonSerializer(entityObject.GetType());
                    entityObject = (object)deserializer.ReadObject(ms);
                    }
                    
            }
            catch (Exception ex){ throw ex; }
            return entityObject;
        }

Retrieve Multiple records using ODATA API-

  /// <summary>
        /// Retrieve multiple using Fetch XML in API Call
        /// </summary>
        /// <param name="entityPluralName">entity name</param>
        /// <param name="fetchXML">fetchXML string</param>
        /// <returns>List of retrieved Objects</returns>
        public List<RetrieveOpportunity> RetrieveMultiple(string entityPluralName, string fetchXML)
        {
            fetchXML = fetchXML.Replace("<fetch", "<fetch {0} ");
            int page = 1;
            var cookie = string.Format("page='{0}'", page);
            string xml = string.Format(fetchXML, cookie);
            string crmRestQuery = apiUrl + entityPluralName + "?fetchXml=" + xml;
            List<RetrieveOpportunity> lstOpties = new List<RetrieveOpportunity>();
            bool isDone = false;
            try
            {
                while (isDone == false)
                {
                    isDone = true;
                    HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, crmRestQuery);
                    //add header parameters
                    request.Headers.Add("Prefer", "odata.include-annotations=\"*\"");//for formatted values
                    request.Headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/XML"));
                    request.Headers.Add("Authorization", "Bearer " + oauthToken);

                    //send request
                    HttpResponseMessage response = httpClient.SendAsync(request).Result;
                    string responseString = response.Content.ReadAsStringAsync().Result;
                    responseString = FormatResponse(responseString);

                    using (var ms = new MemoryStream(Encoding.Unicode.GetBytes(responseString)))
                    {
                        // Deserialization from JSON  
                        DataContractJsonSerializer deserializer = new DataContractJsonSerializer(typeof(APIResult));
                        APIResult apiResult = (APIResult)deserializer.ReadObject(ms);

                        if (!string.IsNullOrEmpty(apiResult.Microsoft_Dynamics_CRM_fetchxmlpagingcookie))
                        {
                            isDone = false;
                            page++;

                            //retrieve pagingCookie value
                            string xmlVal = WebUtility.UrlDecode((WebUtility.UrlDecode((string)apiResult.Microsoft_Dynamics_CRM_fetchxmlpagingcookie)));
                            int Start = xmlVal.IndexOf("pagingcookie=\"", 0) + "pagingcookie=\"".Length;
                            int End = xmlVal.IndexOf("\" istracking=\"False\"", Start);

                            //proccess the pagingCookie value to support in API URL
                            string strVal = xmlVal.Substring(Start, End - Start).Replace("<", "%26lt;").Replace(">", "%26gt;").Replace("\"", "%26quot;").Replace("'", "%26apos;").Replace("&", "%26amp;");
                            cookie = string.Format("page='{0}' paging-cookie='{1}'", page, strVal);

                            //modify the query for next page with paging cookie
                            xml = string.Format(fetchXML, cookie);
                            crmRestQuery = apiUrl + entityPluralName +"?fetchXml=" + xml;

                        }
                        if (!string.IsNullOrEmpty(apiResult.odata_nextLink))
                        {
                            isDone = false;
                            crmRestQuery = apiResult.odata_nextLink;
                        }

                        if (lstOpties == null || lstOpties.Count == 0)
                        {
                            lstOpties = apiResult.value;
                        }
                        else
                        {
                            lstOpties.AddRange(apiResult.value);
                        }
                    }
                }
            }
            catch (Exception ex) {
                throw ex;
            }
            return lstOpties;
        }

        public List<RetrieveOpportunity> RetrieveMultiple(string entityPluralName, string selectColumns,string filterConditions=null, string orderby=null,bool returnCount=false)
        {
           string query= "?$select=" + selectColumns;
            if (filterConditions != null) {
                query += "&$filter=" + filterConditions;
            }
            if (orderby != null) {
                query += "&$orderby=" + orderby;
            }
            if (returnCount == true) {
                query += "&$count=true";
            }

            string crmRestQuery = apiUrl + entityPluralName +  query;
            List<RetrieveOpportunity> lstOpties = new List<RetrieveOpportunity>();
            bool isDone = false;
            try
            {
                while (isDone == false)
                {
                    isDone = true;
                    HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, crmRestQuery);
                    //add header parameters
                    request.Headers.Add("Prefer", "odata.include-annotations=\"*\"");//for formatted values
                    request.Headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/XML"));
                    request.Headers.Add("Authorization", "Bearer " + oauthToken);

                    //send request
                    HttpResponseMessage response = httpClient.SendAsync(request).Result;
                    string responseString = response.Content.ReadAsStringAsync().Result;
                    responseString = FormatResponse(responseString);

                    using (var ms = new MemoryStream(Encoding.Unicode.GetBytes(responseString)))
                    {
                        // Deserialization from JSON  
                        DataContractJsonSerializer deserializer = new DataContractJsonSerializer(typeof(APIResult));
                        APIResult apiResult = (APIResult)deserializer.ReadObject(ms);
                        
                        if (!string.IsNullOrEmpty(apiResult.odata_nextLink))
                        {
                            isDone = false;
                            crmRestQuery = apiResult.odata_nextLink;
                        }

                        if (lstOpties == null || lstOpties.Count == 0)
                        {
                            lstOpties = apiResult.value;
                        }
                        else
                        {
                            lstOpties.AddRange(apiResult.value);
                        }

                    }
                }
            }
            catch (Exception ex)
            {
                throw ex;
            }
            return lstOpties;
        }

Some common Methods in above Code –

  #region Local variables
        /// <summary>
        /// Local Variables
        /// </summary>
        private string clientID, secretKey, resource, oauthToken, apiUrl, tokenURL;
        private HttpClient httpClient;

        /// <summary>
        /// Constructor to initialize the variables
        /// </summary>
        public Operations()
        {
            apiUrl = "https://mscrm16tech.crm8.dynamics.com/api/data/v8.2/";
            clientID = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx";
            secretKey = "asdasdxzxczdsdaasda";
            resource = "https://mscrm16tech.crm8.dynamics.com";
            tokenURL = "https://login.microsoftonline.com:443/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/oauth2/token/";
        }
        #endregion 
 #region Generate Bearer Token
        /// <summary>
        /// Function to generate Bearer token OAuth2.0 using client id and secret
        /// </summary>
        /// <returns>access token</returns>
        public string GetBearerToken()
        {
            oauthToken = null;
            HttpClientHandler clientHandler = new HttpClientHandler();
            clientHandler.Proxy = WebRequest.GetSystemWebProxy();
            clientHandler.Proxy.Credentials = CredentialCache.DefaultCredentials; // or new NetworkCredential("username","password","DOMAIN");
            clientHandler.UseProxy = true;
            httpClient = new HttpClient(clientHandler);
            ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
            HttpContent requestContent = new FormUrlEncodedContent(new Dictionary<string, string>
            {
                {"grant_type","client_credentials"},
                {"client_id",clientID},
                {"client_secret",secretKey},
                {"resource",resource}
            }
            );
            HttpResponseMessage response = httpClient.PostAsync(tokenURL, requestContent).Result;
            String responseString = response.Content.ReadAsStringAsync().Result;
            using (var ms = new MemoryStream(Encoding.Unicode.GetBytes(responseString)))
            {
                // Deserialization from JSON  
                DataContractJsonSerializer deserializer = new DataContractJsonSerializer(typeof(BearerToken));
                BearerToken token = (BearerToken)deserializer.ReadObject(ms);
                oauthToken = token.access_token;
            }

            return oauthToken;
        }
        #endregion
#region Common Methods
       public string FormatResponse(string responseString) {
            responseString = responseString.Replace("@odata.etag", "odata_etag");
            responseString = responseString.Replace("@OData.Community.Display.V1.FormattedValue", "_OData_Community_Display_V1_FormattedValue");
            responseString = responseString.Replace("@Microsoft.Dynamics.CRM.associatednavigationproperty", "_Microsoft_Dynamics_CRM_associatednavigationproperty");
            responseString = responseString.Replace("@Microsoft.Dynamics.CRM.lookuplogicalname", "_Microsoft_Dynamics_CRM_lookuplogicalname");
            responseString = responseString.Replace("@Microsoft.Dynamics.CRM.lookuplogicalname", "_Microsoft_Dynamics_CRM_lookuplogicalname");
            responseString = responseString.Replace("@odata.context", "odata_context");
            responseString = responseString.Replace("@Microsoft.Dynamics.CRM.totalrecordcount", "Microsoft_Dynamics_CRM_totalrecordcount");
            responseString = responseString.Replace("@Microsoft.Dynamics.CRM.totalrecordcountlimitexceeded", "Microsoft_Dynamics_CRM_totalrecordcountlimitexceeded");
            responseString = responseString.Replace("@Microsoft.Dynamics.CRM.fetchxmlpagingcookie", "Microsoft_Dynamics_CRM_fetchxmlpagingcookie");
            responseString = responseString.Replace("@Microsoft.Dynamics.CRM.morerecords", "Microsoft_Dynamics_CRM_morerecords");
            responseString = responseString.Replace("@odata.nextLink", "odata_nextLink");
            return responseString;
        }
        #endregion

Sample Class for Json Serialization in API Calls-

This class is not needed in you are using Newtonsoft.Json library.

using System.Collections.Generic;
using System.Runtime.Serialization;

namespace ODATA_API
{
    [DataContract]
    public class BearerToken
    {
        [DataMember]
        public string token_type;
        [DataMember]
        public string expires_in;
        [DataMember]
        public string ext_expires_in;
        [DataMember]
        public string expires_on;
        [DataMember]
        public string not_before;
        [DataMember]
        public string resource;
        [DataMember]
        public string access_token;
    }

    [DataContract]
    public class CreateUpdateOpportunity {
        [DataMember]
        public string name;
        [DataMember]
        public string estimatedclosedate;
        [DataMember]
        public string parentaccountid_odata_bind;
        [DataMember]
        public decimal estimatedvalue;
        [DataMember]
        public string transactioncurrencyid_odata_bind;
    }
    [DataContract]
    public class RetrieveOpportunity
    {
        [DataMember]
        public string _odata_context;
        [DataMember]
        public string _odata_etag;
        [DataMember]
        public string estimatedclosedate_OData_Community_Display_V1_FormattedValue;
        [DataMember]
        public string estimatedclosedate;
        [DataMember]
        public string estimatedvalue_OData_Community_Display_V1_FormattedValue;
        [DataMember]
        public string estimatedvalue;
        [DataMember]
        public string estimatedvalue_base_OData_Community_Display_V1_FormattedValue;
        [DataMember]
        public string estimatedvalue_base;
        [DataMember]
        public string name;
        [DataMember]
        public string _parentaccountid_value_OData_Community_Display_V1_FormattedValue;
        [DataMember]
        public string _parentaccountid_value;
        [DataMember]
        public string opportunityid;
        [DataMember]
        public string _transactioncurrencyid_value_OData_Community_Display_V1_FormattedValue;
        [DataMember]
        public string _transactioncurrencyid_value;
    }

    [DataContract]
    public class APIResult
    {
        [DataMember]
        public string odata_context;
        [DataMember]
        public int Microsoft_Dynamics_CRM_totalrecordcount;
        [DataMember]
        public bool Microsoft_Dynamics_CRM_totalrecordcountlimitexceeded;
        [DataMember]
        public string Microsoft_Dynamics_CRM_fetchxmlpagingcookie;
        [DataMember]
        public bool Microsoft_Dynamics_CRM_morerecords;
        [DataMember]
        public string odata_nextLink;
        [DataMember]
        public List<RetrieveOpportunity> value;
    }

}

Calling the Above methods Sample-

static void Main(string[] args)
        {
            Operations ops = new Operations();
            Console.WriteLine("Started!!");
            Console.WriteLine(ops.GetBearerToken());
            
            CreateUpdateOpportunity entity = new CreateUpdateOpportunity();

            entity.transactioncurrencyid_odata_bind = "/transactioncurrencies(c63ba581-6bc6-e811-a96f-000d3af04fb0)";
            entity.estimatedvalue = 1000;
            entity.estimatedclosedate = "2021-01-01";
            entity.name = "Generated From C# Code";
            entity.parentaccountid_odata_bind = "/accounts(3b3b7c71-61d2-ea11-a813-000d3af0205e)";
            //create record
            string recordGuid = ops.CreateAPI("opportunities", entity);
            Console.WriteLine(recordGuid);
           
            //update record
            entity.name = "Generated From CSharp Code";
            Console.WriteLine(ops.UpdateAPI("opportunities", recordGuid, entity));

            //retrieve record
            RetrieveOpportunity retrieveOpportunity = new RetrieveOpportunity();
            retrieveOpportunity =(RetrieveOpportunity)ops.RetrieveSingle("opportunities", recordGuid,retrieveOpportunity);
            Console.WriteLine(retrieveOpportunity.name);

            //delete Record
            Console.WriteLine(ops.DeleteAPI("opportunities", recordGuid));


            Console.Read();
        }

Hope this will help…
Enjoy Ms CRM!!!

Follow on Facebook- FB: MSCRM16Tech

MS Dynamics Rest OData API with bearer token(OAuth2.0) using Script Component in SSIS

What is Issue?

There are many connectors available to connect MS Dynamics 365 CRM online in SSIS but all are having their own limitations and licensing constraints while using. So how can we avoid using these connectors and utilize what is available in SSIS by default?

The Reason-

SSIS provides connection managers to connect all available databases but when it comes for MS Dynamics 365 CRM Online, you don’t have any database in your control. So the other way remain is the Rest API call. But there is also no Rest API Connector by default available in SSIS.

Solution-

The solution for this issue is script component available in SSIS. It provides you the scripting ability and you can do what ever you want with script.

I am using the same to connect the MS Dynamics 365 CRM Online OData Rest API with OAuth 2.0 authentication method.

Let’s go step by step-

  1. Add Script component in Data Flow and select as the Source-

2. Double click on Script Component ->”input and Output” and Add some output columns with some proper data types, so that you can add data in output column fetched from API like below-

3. After adding output column click on “Script” on left hand side menu and click on Edit script-

4. It will open another visual studio window with “VstaProject”-
Add the references –
1. Microsoft.CSharp
2. System.Net
3. System.Net.Http
4. System.Web.Extensions

5. Now open the main.cs and add below sample code and change the variables and queries as per your need-
(Below is the example to fetch “estimatedclosedate” and “opportuinityid” from opportunity entity using Rest API.)

#region Help:  Introduction to the Script Component
/* The Script Component allows you to perform virtually any operation that can be accomplished in
 * a .Net application within the context of an Integration Services data flow.
 *
 * Expand the other regions which have "Help" prefixes for examples of specific ways to use
 * Integration Services features within this script component. */
#endregion

#region Namespaces
using System;
using System.Collections.Generic;
using System.Data;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Web.Script.Serialization;
#endregion

/// <summary>
/// This is the class to which to add your code.  Do not change the name, attributes, or parent
/// of this class.
/// </summary>
[Microsoft.SqlServer.Dts.Pipeline.SSISScriptComponentEntryPointAttribute]
public class ScriptMain : UserComponent
{
    #region Help:  Using Integration Services variables and parameters
    /* To use a variable in this script, first ensure that the variable has been added to
     * either the list contained in the ReadOnlyVariables property or the list contained in
     * the ReadWriteVariables property of this script component, according to whether or not your
     * code needs to write into the variable.  To do so, save this script, close this instance of
     * Visual Studio, and update the ReadOnlyVariables and ReadWriteVariables properties in the
     * Script Transformation Editor window.
     * To use a parameter in this script, follow the same steps. Parameters are always read-only.
     *
     * Example of reading from a variable or parameter:
     *  DateTime startTime = Variables.MyStartTime;
     *
     * Example of writing to a variable:
     *  Variables.myStringVariable = "new value";
     */
    #endregion

    #region Help:  Using Integration Services Connnection Managers
    /* Some types of connection managers can be used in this script component.  See the help topic
     * "Working with Connection Managers Programatically" for details.
     *
     * To use a connection manager in this script, first ensure that the connection manager has
     * been added to either the list of connection managers on the Connection Managers page of the
     * script component editor.  To add the connection manager, save this script, close this instance of
     * Visual Studio, and add the Connection Manager to the list.
     *
     * If the component needs to hold a connection open while processing rows, override the
     * AcquireConnections and ReleaseConnections methods.
     * 
     * Example of using an ADO.Net connection manager to acquire a SqlConnection:
     *  object rawConnection = Connections.SalesDB.AcquireConnection(transaction);
     *  SqlConnection salesDBConn = (SqlConnection)rawConnection;
     *
     * Example of using a File connection manager to acquire a file path:
     *  object rawConnection = Connections.Prices_zip.AcquireConnection(transaction);
     *  string filePath = (string)rawConnection;
     *
     * Example of releasing a connection manager:
     *  Connections.SalesDB.ReleaseConnection(rawConnection);
     */
    #endregion

    #region Help:  Firing Integration Services Events
    /* This script component can fire events.
     *
     * Example of firing an error event:
     *  ComponentMetaData.FireError(10, "Process Values", "Bad value", "", 0, out cancel);
     *
     * Example of firing an information event:
     *  ComponentMetaData.FireInformation(10, "Process Values", "Processing has started", "", 0, fireAgain);
     *
     * Example of firing a warning event:
     *  ComponentMetaData.FireWarning(10, "Process Values", "No rows were received", "", 0);
     */
    #endregion

    private String clientID, secretKey, resource, oauthToken, apiUrl, tokenURL;
    private HttpClient httpClient;

    /// <summary>
    /// This method is called once, before rows begin to be processed in the data flow.
    ///
    /// You can remove this method if you don't need to do anything here.
    /// </summary>
    public override void PreExecute()
    {
        base.PreExecute();
        /*
         * Add your code here
         */
        apiUrl = "https://xxxxxx.crm8.dynamics.com/api/data/v8.2/";
        clientID = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx"; // Client id to generate Bearer Token
        secretKey = "xxxxxxxxxxxxxxxxxxxx"; //Secret Key
        resource = "https://xxxxxx.crm8.dynamics.com"; //resource
		tokenURL="https://login.microsoftonline.com:443/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxxx/oauth2/token/"
        GetBearerToken();
    }

    private void GetBearerToken()
    {
        HttpClientHandler clientHandler = new HttpClientHandler();
        clientHandler.Proxy = WebRequest.GetSystemWebProxy();
        clientHandler.Proxy.Credentials = CredentialCache.DefaultCredentials; // or new NetworkCredential("username","password","DOMAIN");
        clientHandler.UseProxy = true;
        httpClient = new HttpClient(clientHandler);
        ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12;
        HttpContent requestContent = new FormUrlEncodedContent(new Dictionary<string, string>
            {
                {"grant_type","client_credentials"},
                {"client_id",clientID},
                {"client_secret",secretKey},
                {"resource",resource}
            }
        );
        HttpResponseMessage response = httpClient.PostAsync(tokenURL, requestContent).Result;

        String responseString = response.Content.ReadAsStringAsync().Result;

        JavaScriptSerializer serialiser = new JavaScriptSerializer();
        dynamic apiResult = serialiser.DeserializeObject(responseString);

        oauthToken = (String)apiResult["access_token"];
    }

    /// <summary>
    /// This method is called after all the rows have passed through this component.
    ///
    /// You can delete this method if you don't need to do anything here.
    /// </summary>
    public override void PostExecute()
    {
        base.PostExecute();
        /*
         * Add your code here
         */
    }

    public override void CreateNewOutputRows()
    {
        /*
          Add rows by calling the AddRow method on the member variable named "<Output Name>Buffer".
          For example, call MyOutputBuffer.AddRow() if your output was named "MyOutput".
        */
        String query = "$select=opportunityid,estimatedclosedate";
        String crmRestQuery = apiUrl + "opportunities?" + query;

        Boolean isDone = false;
        while (isDone == false)
        {
isDone = true;
            HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, crmRestQuery);

            //add header parameters
            request.Headers.Add("Prefer", "odata.include-annotations=\"*\"");//for formatted values
            request.Headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
            request.Headers.Add("Authorization", "Bearer " + oauthToken);

            //send request
            HttpResponseMessage response = httpClient.SendAsync(request).Result;
            String responseString = response.Content.ReadAsStringAsync().Result;

            JavaScriptSerializer serialiser = new JavaScriptSerializer();
            serialiser.MaxJsonLength = 50 * 1000000;	// Increased to deserialize large number of fields 
            dynamic apiResult = serialiser.DeserializeObject(responseString);

            //--------process dynamic apiResult

            object[] result = null;
            //get only values from response
            foreach (var item in apiResult)
            {
                object key = item.Key;
                if ((string)key == "value")
                    result = (object[])item.Value;
              if ((string)key == "@odata.nextLink")
                {
                    isDone = false;
                    crmRestQuery = (string)item.Value;
                }
            }
            //array of object to list
            List<object> res = result.ToList<object>();
            //for each record in result
            foreach (var item in res)
            {
                //add row of output columns for each record
                Output0Buffer.AddRow();

                Dictionary<string, object> obj = (Dictionary<string, object>)item;
                //all attributes from Record (object)
                foreach (KeyValuePair<string, object> item1 in obj)
                {
                    if (item1.Key == "opportunityid")
                        Output0Buffer.ID = (String)item1.Value;
                    if (item1.Key == "estimatedclosedate")
                        Output0Buffer.EstCloseDate = (String)item1.Value;
   if (item1.Key == "estimatedclosedate@OData.Community.Display.V1.FormattedValue")
                        Output0Buffer.EstCloseDateFormatted = (String)item1.Value;
                }
            }
        }
    }
}

In case of API using FetchXML you can refer the below code-
We can use pagingcookie for 5000+ records

    public override void CreateNewOutputRows()
    {
        /*
          Add rows by calling the AddRow method on the member variable named "<Output Name>Buffer".
          For example, call MyOutputBuffer.AddRow() if your output was named "MyOutput".
        */
        string query = @"<fetch {0} version='1.0' output-format='xml-platform' mapping='logical' distinct='false'>
                                         <entity name='opportunity'>
                                            <attribute name='opportunityid' />
                                            <attribute name='estimatedclosedate' />
                                          </entity>
                                        </fetch>";

        int page = 1;
        var cookie = string.Format("page='{0}'",page);

        string xml = string.Format(query, cookie);
        string crmRestQuery = apiUrl + "opportunities?fetchXml=" + xml;

        bool isDone = false;
        while (isDone == false)
        {
            isDone = true;
            HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Get, crmRestQuery);
            //add header parameters
            request.Headers.Add("Prefer", "odata.include-annotations=\"*\"");//for formatted values
            request.Headers.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
            request.Headers.Add("Authorization", "Bearer " + oauthToken);

            //send request
            HttpResponseMessage response = httpClient.SendAsync(request).Result;
            string responseString = response.Content.ReadAsStringAsync().Result;

            JavaScriptSerializer serialiser = new JavaScriptSerializer();
            serialiser.MaxJsonLength = 50 * 1000000;	// Increased to deserialize large number of fields 
            dynamic apiResult = serialiser.DeserializeObject(responseString);

            //--------process dynamic apiResult

            object[] result = null;
            //get only values from response
            foreach (var item in apiResult)
            {
                object key = item.Key;
                if ((string)key == "value")
                    result = (object[])item.Value;
                //when Rest API using fetchXML 
                if ((string)key == "@Microsoft.Dynamics.CRM.fetchxmlpagingcookie")
                {
                    isDone = false;
                    page++;

                   //retrieve pagingCookie value
                   string xmlVal = WebUtility.UrlDecode((WebUtility.UrlDecode((string)item.Value)));
                   int Start = xmlVal.IndexOf("pagingcookie=\"", 0) + "pagingcookie=\"".Length;
                   int End = xmlVal.IndexOf("\" istracking=\"False\"", Start);
                   
                    //proccess the pagingCookie value to support in API URL
                    string strVal= xmlVal.Substring(Start, End - Start).Replace("<", "%26lt;").Replace(">", "%26gt;").Replace("\"", "%26quot;").Replace("'", "%26apos;").Replace("&", "%26amp;");
                    cookie = string.Format("page='{0}' paging-cookie='{1}'",page, strVal);
                    
                    //modify the query for next page with paging cookie
                    xml = string.Format(query, cookie);
                    crmRestQuery = apiUrl + "opportunities?fetchXml=" + xml;

                }

            }
            //array of object to list
            List<object> res = result.ToList<object>();
            //for each record in result
            foreach (var item in res)
            {
                //add row of output columns
                Output0Buffer.AddRow();

                Dictionary<string, object> obj = (Dictionary<string, object>)item;
                //all attributes from Record (object)
                foreach (KeyValuePair<string, object> item1 in obj)
                {
                    if (item1.Key == "opportunityid")
                        Output0Buffer.ID = (String)item1.Value;
                    if (item1.Key == "estimatedclosedate")
                        Output0Buffer.EstCloseDate = (String)item1.Value;
                    if (item1.Key == "estimatedclosedate@OData.Community.Display.V1.FormattedValue")
                        Output0Buffer.EstCloseDateFormatted = (String)item1.Value;
                }

            }
        }

6. Build the VstaProject and close the VstaProject Visual Studio window.

7. Click on Ok button in Script Component Properties window.

8. Save the SSIS project and execute the Data flow Task.

9. When you add Data viewer on output of Script component, you will get desired output-

I used client_credentials method to generate Bearer Token, you can use the other methods and modify GetBearerToken() method.

The all the parameters I used as hardcoded, you can manage it by passing as input/output parameters.

Hope this will help…
Enjoy Ms CRM!!!

Follow on Facebook- FB: MSCRM16Tech