https://community.workday.com/node/577006?

check_logged_in=1

Skip to main content

Skip to Main Content

Workday Community
Search

 Learn
 Products
 Collaborate
 Feature Release
 Get Help
Breadcrumb
1. Home
2. Parallel Paged Get Component

Parallel Paged Get Component

Primary tabs
 VIEW (ACTIVE TAB)
 REVISIONS

Posted Oct 8, 2019Updated Apr 20, 2022Retirement date: May 11, 2024Read 1288 times

STOP: DON'T SHARE THIS PAGE WITH CUSTOMERS. ACCESS TO THIS CONTENT

IS LIMITED TO PROFESSIONAL SERVICES.

Tool Type Tool / Template

Using Workday Custom Integrations & Apps

Product Integrations

Audience Integration Consultant

 Services Product Lead Area Integrations

The parallel paged get component is used to call Workday's SOAP API's that support
paging. The component will call each page in sequence and can automatically aggregate
all the response pages together.

Forming the web service request

APIs which support paging will have a request containing :  

<wd:Response_Filter>

<wd:As_Of_Effective_Date>[Current Date]</wd:As_Of_Effective_Date>

<wd:As_Of_Entry_DateTime>[Current Date and

Time]</wd:As_Of_Entry_DateTime>

<wd:Page>1</wd:Page>

<wd:Count>100</wd:Count>

</wd:Response_Filter>

The Count property allows you to specify how many records to return on each page. The
Page property specifies which page of data to return. When using the paged get
component, you do not need to make multiple calls to retrieve each page. The component
will do it for you. Some APIs will cache several pages at once when retrieving the first
page. This will only happen when each page request is identical. Therefore, because the
wd:As_Of_Entry_DateTime property defaults to the current moment, it will have a
different value for each page. You must specify a time for the entry moment field in your
initial request to take advantage of the caching.

Retrieving Pages in Parallel

When the  is.paged.get.parallel property is set to true, the  component will fetch each
pages in parallel using four threads, meaning four pages are retrieved simultaneously.  
The pages are retrieved as follows :

 First page on it’s own thread for the PagedGet component

 Page 2 – retrieved by thread 1
 Page 3 – retrieved by thread 2
 Page 4 – retrieved by thread 3
 Page 5 – retrieved by thread 4
 Page 6 – retrieved by first thread which completes the retrieval of pages 2-5
 Page 7 – retrieved by the second thread which completes the retrieval of pages 2-
6
 Etc.

Aggregation

The following properties can be set to allow the pages to be automatically aggregated :

Property Description Typica

is.paged.get.aggregate.header root element of the aggregated message '<root>

 Property Description Typica

is.paged.get.aggregate.footer Closing root element of the aggregated message '</root>

is.paged.get.aggregate.xpath Optional xpath to specify which part of the message to 'env:En

aggregate

Performing aggregation in this way works well for small to medium message sizes. The
paged get component will use either memory or  file backed managed data to keep track
of the aggregation. This is all controlled automatically by the component.

Using local-in
However, for maximum efficiency, the paged get component can be configured to call a
local-in for each page as well as a local-in to control the aggregation.

is.paged.get.process.endpoint is set to the name of a local-in  which will process each

page of the API response.

is.paged.get.parallel.aggregation.endpoint is set to the name of the local-in which will

control aggregation of the processed pages. It is called automatically after each call
to is.paged.get.process.endpoint. 

Two built in properties,  props['aggregator.destination.force.collate']

and props['aggregator.destination.force.batch'] are automatically populated to indicate if
aggregation is collating or batching.

Variables
When using multi-threading in Studio, each thread is passed a copy of the mediation (i.e.
the current message plus all variables and properties). Because the thread is running in
its own space, it can read from its copy of variables and properties but it cannot write to
them. The exception is any variable whose name starts with 'c2p' (child 2 parent). These
variables can be written to and then read by the aggregation or calling thread.

Therefore, in order to pass the output of the paging local-in to the aggregation local-in,  
the output of the local-in must be written to a 'c2p' variable.  

Sample Code
In the clar attached to this page, a call is made to Get_Workers. A paged get component
is used to process each page in a local-in, "HandleOnePage".

The HandleOnePage local-in transforms each page of the response and sets the output
to a variable called "c2p.PageFileOutbound".  
The aggregation endpoint :

The copy step, copies c2p.PageFileOutbound to the message so that it can be

aggregated. This is only performed when  props['aggregator.destination.force.collate'] is
true. The batch when and collate when properties of the aggregator are set
to props['aggregator.destination.force.batch']
and props['aggregator.destination.force.collate'] respectively.

The aggregation is running in its own thread. In order to pass back data from this thread
to the main calling thread, again a c2p variable is needed.
 

When control is passed back from the paged get component, a store step is used to write
a file from the c2p.agg.message variable. Obviously, it will depend on the use case, but it
may be more efficient to process the aggregated file after the aggregator. The c2p
variable is only needed if processing is needed from the main assembly thread.

Attachments
Get_Parallel_Page_Demo_2.clar

Get_Parallel_Page_Demo_2
.clar

Ask a related question

Was this content helpful?

Your feedback will help us improve the content on Workday Community.

Share Your Feedback

Subscriptions

Manage all My Subscriptions Subscribe







 Privacy

|

 Legal

|
 Cookie Preferences

|

 Contact Us

|

 Workday Careers

|

 Community Guidelines

|

 © 2022 Workday, Inc.