LEVEL 2 – LEVEL 7 PROMOTION DOCUMENT

EFFECTIVE
EMPLOYEE NAME: Prerna Suneja CURRENT JOB TITLE: SDE - I Q2
QUARTER:
TIME IN LEVEL
MANAGER NAME: Ankur Gupta PROPOSED JOB TITLE: SDE - II 23
IN MONTHS:
Software
PROMOTION
STEAM MEMBER: Dave Clark CURRENT BUSINESS TITLE: Development Agrim Jain
APPROVER:
Engineer
Software
STEAM DIRECT: Dave Bozeman PROPOSED BUSINESS TITLE: Development
Engineer

FEEDBACK PROVIDERS

Select up to six feedback providers who are at or above the proposed promotion level. Learn feedback best practices.
FEEDBACK PROVIDER BUSINESS TITLE, LEVEL STEAM RELATIONSHIP SUPPORT? DATE
1 Subhadeep Gupta Sr. SDE, L6 Dave Clark Peer SDE Yes 5/30/22
2 Ruchit Shah Sr. Manager - PM, L7 Dave Clark PM Stakeholder Yes 5/26/22
3 Ajay Aneja SDE-II, L5 Dave Clark Peer SDE Yes 5/24/22
4 Ayushi Mittal SDE-II, L5 Dave Clark Peer SDE Yes 5/25/22
5 Abhishek Saroha SDE-II, L5 Dave Clark Peer SDE Yes 5/24/22
Mohsen
6 Sr. Applied Scientist Dave Clark Research Stakeholder Yes 5/18/22
Moarefdoost
Sudheer
7 Principal PM, L7 Dave Clark PM Stakeholder Yes 5/24/22
Pamidighantam

SCOPE OF ROLE
The scope of role section should be written as it would apply to any person in this role. The scope of role should accomplish two
things: Describing their current role responsibilities, complexity, ambiguity, level of influence required, and impact; and defining
the scope of their role at the next level. Learn more about writing the scope of role. (500 words or fewer are recommended)
Prerna Suneja is part of Relay Load-Board Pricing (ALPS) team, part of Relay Product and Tech (RPT) organization as SDE I.
Relay Load-Board (RLB) allow carriers to view, search and accept adhoc loads at dynamic price. We own dynamic pricing
systems to provide carriers competitive price for loads that they can accept on load board. Our system is based on pricing
models which are responsive to marketplace data, seasonality and feedback. We experiment and evolve pricing models as we
learn. Continuous supply of loads at competitive pricing in our network creates a flywheel effect to attract more carriers to
stay or sign up with Amazon, eliminating supply constraints.
To be an SDE II in ALPS team, s/he is required to focus and deliver on design, development/implementation and quality of
complex and ambiguous business problems. As, our team rely heavily on data, machine learning (ML) algorithms and
experimentation, s/he is required to be able to bring alignment among tech and non-tech stakeholders (e.g. - PM, Research,
Operations) to define, deep-dive and deliver configurable solutions. In the absence of an active TPM support, SDE II be
expected to lead project discussions with Tech and business stakeholders as needed. For this, the person would be required to
communicate ideas effectively, recognize discordant views and take part in constructive dialogue to resolve them. To deliver
items planned in our roadmap, SDE-II needs to be capable of working independently and owning solutions end-2-end. SDE-II in
the team is expected to produce high quality work and ensuring that same high bar is maintained while doing code reviews.
As, we own ML code base as well backend, SDE-II should be able to work on ML scripts and do code-review raised by Research
team with minimal guidance.
Due to critical nature of Relay Load-Board, SDE-II design is required to simple and scalable solution, and implementation
should to be defect free, have fallbacks to ensure high availability and must ensure backward compatibility. To ensure best
customer experience and improved dev efficiency, SDE-II is expected to proactively work on OE items and resolve recurring
issues. SDE-II needs to be able to define, set and adopt best practices in coding, design and other dev activities (metrics
analysis, deployment, debugging and experimentation etc.) to improve team efficiency and software quality.
As an SDE-II, s/he should be able to confidently train new team-mates about your customers, what his team’s software does,
how it is constructed, tested, operates, and how it fits into the bigger picture.

PROMOTION ASSESSMENT

Amazon Confidential v2022.04 1

LEVEL 2 – LEVEL 7 PROMOTION DOCUMENT
Describe how the employee has performed at the next level; provide examples of how they demonstrate our Leadership
Principles. Use examples that illustrate this person’s body of work, specifically key performance highlights and misses.

Describe their ability to Hire and Develop the Best (Leaders raise the performance bar with every hire and promotion. They
recognize exceptional talent, and willingly move them throughout the organization. Leaders develop leaders and take seriously
their role in coaching others.). Include contributions in support of our commitment to diversity and inclusion.

If this person is a people manager: When reviewing the past year, are there any Connections findings that require explanation?
Learn more about writing the promotion assessment. (1500 words or fewer are recommended)
SUMMARY
Prerna joined Amazon as an SDE-Intern in Jan’20, joined as SDE-1 in Jul’20 in Amazon Exports and Expansions and moved to
RLB Pricing (ALPS team) in Jan’21. In last 2 years, Prerna has been able to deliver various complex ambiguous projects - MASS
(unified guardrails), Trips Model Migration from Random Forest to XGBoost, RLB Pricing Simulator (vRLB), RLB launch in
Canada (design), Trips and Block Guardrails based on RCM (design) and Pricing Anomaly Detection (in-
progress, design) independently with minimal guidance. She has worked on almost all components/services (both machine-
learning and tech components) owned by our team and raises bar in ownership, bias for action and deliver results. She has
been able to deep-dive on new initiatives and always comes up various approaches with detailed pros/cons, which is reflected
in various design/tech documents. She excels in deep-dive to solve operational issues with-in the team like integration test-
case for pricing service (wiki1, wiki2), high-sev reduction (wiki) and various high/low severity tickets. Apart from this, she has
drove multiple launch initiatives with product/research team, drove various deep-dive sessions (guardrails anomalies, cost
deviations), raises the bar on insist on high standards with MCM templates and also helped various team-members and is
currently mentoring an intern on his onboarding and project delivery.

COMPLEX DELIVERY, DESIGN, INDEPENDENT EXECUTION

MARKET AWARE SEARCH SPACE (FORMERLY UNIFIED GUARDRAILS) & ANOMALY DETECTION (APR’21-MAR’22)
RLB Pricing used to maintain a separate set of static model guardrails, which was passed as a search space to the ML Pricing
Framework, where our ML Pricing Framework would first predict a price that a carrier is most likely to accept, at the lowest
cost, then business guardrails were applied as a post-processing event. MASS allows us to simplify our tech stack by collapsing
the model and business guardrails into a single set of business guardrails, then feeding those as an input into our ML Pricing
Framework.
Prerna worked on LLD for MASS and detailed out various use-cases with specific examples and proposed recommended
solution. One of the major challenges was to decide the min/max bound to be passed for a lead time as multiple business
guardrails rules could be matching. Based on different deep-dive and analysis for various use-cases and historical data, Prerna
was able to align tech and product team on final solution to utilize maximum of all floor guardrails and minimum of all ceiling
guardrails. Prerna was also suggested new DW changes to store pricing journey to resolve model search space which has been
really helpful in offline deep-dive and also proposed new business metrics (loads priced at lower/upper bounds).
During implementation, Prerna worked in pricing service and model algorithm package to change model input to pass all lead-
time guardrails and get it code-reviewed with research stakeholders. One of the execution challenges was latency due to
model search space computation for all lead-time based on business guardrails. Prerna deep-dived on CPU profiling pattern,
found out various issue in current code structure and also utilized multi-threading to parallelize fetching guardrails for various
lead-times.

In rollout, Prerna showed ownership and deep-diving increased premiums herself due to PM attrition. She identified existing
issues with guardrails and identified guardrails anomalies (which become a new work stream for pricing) and discussed these
with tech/program team to resolve. She relentlessly worked with program managers to resolve guardrails anomalies and also
setup automated daily/weekly on anomalies for visibility. Prerna also gave multiple KT sessions and worked with program
manager to identify guardrails changes based on lower/upper bound hit rate to improve MASS business metrics.

MASS was tech complete on time without any tech issues in production and finally launched in production after continuous
deep-dive for 3 months. MASS improved conversion by 167bps for One-Ways and 76 bps for Tours, improved premium by 375
bps for One-Ways and 197 bps for Tours, while moving us further towards our Hands-off-the-Wheel pricing goal. For India, we
observed similar improvements in pricing automation and an 86-bps improvement in premium (% vs 3P OW). In Wk13’22, we
expanded to 100% of the network for both One-Ways and Trips in the US, where we anticipate $104M in ’22 savings and
$135M in annualized entitlement.

Apart from MASS, based on these guardrails’ anomalies and COE, we identified the needs for Pricing Anomaly Detection
framework. Prerna worked on HLD for Pricing Anomaly Detection. As this is fairly ambiguous problem, Prerna worked on listing
Amazon Confidential v2022.04 2
LEVEL 2 – LEVEL 7 PROMOTION DOCUMENT
down all the framework requirements and prioritized list of anomalies and get it reviewed with team and PM/Research
Stakeholders before working on HLD. After reviewing the requirements and aligning on MLP scope, Prerna deep-dived
thoroughly and evaluated multiple Anomaly Detection strategy used by Walmart and Amazon Retail. In HLD, Prerna detailed
out various Anomaly Detection framework with detailed pros/cons, batch/stream processing, pre-facto and post-facto
detection and outlined the high-level design with new components and their responsibilities. Prerna’s deep-dive and
documentation skills has been appreciated by the team and she has received multiple accolades for the same. The exact
anomaly detection algorithm is still in-progress (requires research deep-dive) and we are planning to launch by 8/30.

AMAP – TRANSIT TIME BASED GUARDRAILS (SEP’21-MAR’22)

For pricing demands on RLB, we use guardrails as a protective mechanism to ensure that price returned by model is neither
too low to protect customer trust and neither too high to save on premiums. Adaptive Marketplace Aware Pricing (AMAP)
aims that our guardrails should be adaptive based on internal/external market rates and trends. As part of AMAP, we started
ingesting DAT data within our pricing systems and define guardrails based on these rates to better react to changes in
marketplace rates. However, DAT rates are based on origin/destination pair and distance, which doesn’t solve the use-case for
transit time guardrails for Trips.

A-MAP trip transit time guardrails project, she broke down the problem into raw data collection, data ingestion and
processing, offline and online workflows. I have seen similar approach in other projects as well. This approach makes sure that
each part of the design has been given proper attention, and hence the design review process also becomes easier for the rest
of the team.

Prerna worked on LLD design for transit time guardrails based on RCM benchmark. She synced with PMs (James D’Amato,
Sudheer Pamidighantam) and tech stakeholders (Ajay Aneja – peer SDE-II) to deep-dive and understand the current set of
guardrails used, use-cases where transit time guardrails are helpful and did thorough data deep-dive on effectiveness on these
guardrails for the different use-cases, demonstrating Learn & Be Curious. As per initial roadmap scope and project
requirements, we need to onboard RCM benchmark rate type to create marketplace aware transit time aware guardrails.
Though, Prerna went above and beyond (Thing Big) and find out the issues in current system like conflicting transit time and
distance band rules, non-linear relationship for transit time and distance, incorrect condition setup for open loop trips etc.

In LLD, Prerna broke down the problem the into guardrails granularity, raw data collection, data ingestion and processing,
offline and online workflows. She methodically details out the LLD for each of the sub-problems, along with pros/cons
wherever required and provided recommendations with proper reasoning (as highlighted in Subhadeep’s feedback). For data
processing, Prerna worked with RCM and Costing tech team to understand RCM DW table schema and came with detailed
proposal on defining/processing RCM rate type to take base rates for only accepted contracts with-in 2 weeks duration. As the
number of RCM contracts are skewed based on the proposed granularity, she ensured to propose two tiers fallback rates for
100% coverage (Insist on High Standards), which wasn’t even present for DAT rate type in Freight Market Rates Service.
Prerna also suggested changes in existing GetMarketRates API to support new rate type and made it extensible, demonstrating
Insist on High Standards and Think Big to support blocks movement as well, which helped in launching RCM based guardrails
for blocks with-in a sprint (not part of roadmap initially). The design was reviewed with-in tech team and PM stakeholders and
received on major call-outs.

After closing LLD, Prerna worked on implementation and ensured to make Freight Marketplace Rates Service (FMRS)
extensible for new rate type addition, as this was second rate type in FMRS. She used right level of abstraction (as highlighted
in Ajay’s feedback), added support for RCM deviation reports and added support for multiple fallback rate types through
configurations only. She ensured her changes are thoroughly tested, backward compatible and didn’t cause any regression
issues. Prerna ensured to write test cases and code coverage velocity of > 95% and made sure to add all necessary metrics in
DW and setup alarms to catch anomalies.

LLD/HLD: RCM Based Guardrails

This has been launched in production on time (Deliver Results) even though the multiple new requirements like fallback
pricing, block pricing etc. were handled (Bias for Action), without any low/high sev issues. This framework is currently being
utilized in NA for Blocks and under experimentation for Trips.

FEEDBACK FROM RABIN PAUL (PRINCIPAL PM, L7) – I have worked with Prerna on block guardrails. She demonstrated a very clear
understanding of how RCM based guardrails worked, and was able to explain the details of how it works in a simple way. She
demonstrated ownership explain bring new teammates upto speed on every detail of how the current pricing guardrails
worked. She demonstrated “Ownership” and “earned trust” through these interactions.
Amazon Confidential v2022.04 3
LEVEL 2 – LEVEL 7 PROMOTION DOCUMENT

RLB PRICING SIMULATOR (FORMERLY VRLB) (JAN’22 – JUN’22)

RLB Pricing Simulator is a simulator that artificially simulates the transportation marketplace environment, carriers’ responses
and behaviors. Similar to a flight simulator, which artificially simulates aircraft flight in various environments and conditions
and allows pilots to practice flying and train for missions, pricing simulator allows scientists, SDEs and business users to test
and evaluate new features and models for RLB. The vRLB will model and simulate transportation marketplace and through that
provides ability for running and assessing multiple scenarios. This simulator can also be used to train Reinforcement Learning
agents, run what-if scenarios, perform cost analysis, and stress-test new initiatives.
Prerna worked with James D’Amato (Product Manager) and Mohsen Moarefdoost (Sr. Applied Scientist) to close out on the
requirements and understood the research doc and analysis. She asked the right set of questions and ensured to understood
various entitles in Pricing Simulator. She did work with research to solidify requirements on pricing simulator specifically –
understanding load and carrier agent, significance of number of replications, conversion and premiums calculation from each
replication and overall and real-time feature value impact.
In HLD, Prerna ensured to detail out requirements for periodic and adhoc simulations and listed motivation in detail after
evaluating gaps in current system. She broke down the problem into simulation workflow, replication workflow and workflow
run environment. She further divided and established clear responsibility for simulations and replication workflows. There
were 4 primary challenges in HLD - a) running each replication workflows independently, b) carrier response prediction c) live
feature calculation/prediction (future scope) and d) workflow environment to run. Prerna deep-dive and detailed out
thoroughly on each of the challenges and showed Invent & Simplify to avoid sagemaker for carrier response validation for MLP
as it wasn’t required. She divided live feature in two broad categories (considering future scope as well) –
computable/deterministic and non-computable and non-deterministic and suggest approach for both of them to ensure
design remain extensible for future.
In LLD, Prerna details out different steps required in step function for simulations workflows, replication workflow and
input/output for both workflows. She also detailed out the integration with shadow environment for automated periodic runs
and integration with manual analysis pipeline for adhoc/manual runs. Prerna’s design wiki has the necessary details and
sections and was approved with-in the team without any major callouts. The design was also approved in cross-org Samurai
group without any major callouts except on research model understanding.
During implementation, Prerna took care of research peer code-review and gave code-review comments. There were two
major challenges during implementation – a) Simulator Accuracy b) Execution Time. Prerna worked with product manager and
research stakeholders to finalize accuracy thresholds for Simulator and ran various (20+) experiments runs to measure
accuracy. As part of these runs, she deep-dive and found out that accuracy was low due to real-time supply aware feature. As
computing live features weren’t part of MLP requirements, she provided an alternate approach to pass approximate supply
aware feature and got alignment with research/PM stakeholders, which improved the accuracy. On execution time, she
ensured did multiple rounds of load-testing to ensure no regression impact and avoid any scalability concerns. She also made
some optimization in price prediction calls specific to Pricing Simulator runs which reduced execution time by ~40%.
Prerna ensured to write test cases and code coverage velocity of > 95% and made sure to add all necessary metrics in DW.
Pricing Simulator was launched in production 2 weeks before proposed schedule (Deliver Results) without any low/high sev
issues. The framework is currently being tested for business metrics (premiums/conversion) in parallel with on-going
production feature experiments to gain confidence before we use it for new/existing feature changes.
LLD/HLD: RLB Pricing Simulator

QUALITY OF WORK
Prerna consistently follows best practices for readability, maintainability, writes unit cases and ensures high code coverage.
She has submitted 163 code reviews in 2021 (highest in the entire team), 60 code reviews in 2022 (second highest in the entire
team) and average number of code review iterations in 2021 is 1.2 (second lowest in the entire team). She has made sure to
have 95%+ unit test coverage in all packages – OptimusPricingService, SibylModelRefreshPipeline and Freight Market Service
packages. Apart from tech component, she has also ensured to maintain high standard on RLBPricingMLAlgorithm by using
modularity, logging and adding utility functions to allow long-term flexibility. Prerna maintains high quality of documentation
in her CR details that help the reviewers to quickly review the CRs. She has been able to defend and is receptive to code-
review comments and ensure the right quality code is delivered. Apart from her code-reviews, Prerna has been active
participant in doing peer’s code-reviews. She has approved 222 code-reviews since 2021 (third-highest in the entire team) and
commented on 218 code-reviews (second highest in the entire team). Some of the code-reviews done by her - CR-57508516,
CR-56738208, CR-65051620. She has pointed out logical issues, maintainability issues, missing test-cases etc. Apart from this,
Prerna is front-runner in identifying issues in the team and creating backlog items (43 in last 1 year) to get it prioritized and she
has even worked with RLB BI team multiple times by creating backlog, providing KTs to fix dashboard for experiment/feature
Amazon Confidential v2022.04 4
LEVEL 2 – LEVEL 7 PROMOTION DOCUMENT
analysis.

ENGINEERING EXCELLENCE, OPERATION EXCELLENCE & OWNERSHIP

On-call – Prerna has resolved 130 high and low sev tickets (second-highest in the team) in last one year. She has been able to
reduce the number of tickets in our queue by 50% during her on-call rotation twice in last 1 year. She has been very proactive
and shows great deep-dive in all on-call issues, have reduced open ticket count < 10 in her all 4 on-call weeks in 2022 and
effectively able to represent her team in weekly OE meetings. She was able to independently root-cause multiple sev-2 and
high impacting tickets and even worked on fixing them during on-call itself like – D43452246 (helped in reduction of p90
latency from 9 seconds to 6 seconds), V570699297 (mis-configured auto-scaling policy), V538354071 (incorrect workflow
validations), V538279673, V571257783 (incorrect alarms thresholds), V495733862 (invalid configurations). Prerna always
ensures to update the ticket correspondence with all necessary details and also ensure to cover preventive action items before
closing any ticket. Prerna has handled on-calls in multiple peak periods. She was able to manage all overrides and guardrails
requests on-time and was proactive on all issues. Her efforts were acknowledged and appreciated by PMs (AJ Stark, Sudheer
and James).
EE Goal - Integration Test Framework - Prerna has been SME for setting up integration test framework for Optimus Pricing
Service (business critical service) in 2021 to detect code-bugs early, reduce deployments rollbacks and sanity testing time. She
created a document, which details about current state, challenges, test cases plan, pipeline automation platform, response
validation and success criteria/goal. As pricing service response can change even for small changes and model updates, it is not
straight forward to verify exact response in integration testcases. Prerna deep-dive and came up with various options for
response validation with detailed pros/cons, also for test framework and her recommendation was approved/aligned with-in
team and cross-org Samurai group as well. She delivered on 2021 integration goal, setup basic infrastructure for integration
test cases validation with P0 test cases and automation in pipelines. This has reduced sanity testing time from 2.5 hours to < 30
mins and reduced faulty deployments < 20%. She also took multiple KTs sessions (online/offline) to help team to deep-dive
and contribute to integration test cases framework. In 2022, she took more aggressive goal to enhance the framework and
reduce faulty deployments to < 10%, she had already documented the new problems and solutions for them along with
pros/cons and is working towards on achieving the goal.
COE – Incorrect prices leading to high premiums - Prerna figured out defect (manually during separate deep-dive), where
pricing service was incorrectly giving high prices. She ensured to highlight the issue at the same time, as it wasn’t detected
automatically and made the immediate fix to avoid customer/business impact. She took the ownership for writing COE (even
though it wasn’t caused due to her defect). She did great work in deep-diving on the root-cause, action-items and
documentation for the COE and got it approved with-in the team, cross-org Samurai group and bar raiser. She also worked on
most of the action items to fix and also worked with BI team to get action items prioritized.
MCM Templates – Prerna worked on re-defining the weekly deployment MCM template for RLB pricing team and get it
reviewed with-in the team and MCM Bar-Raiser. This has reduced high sev issues in production and also reduced time to
create an MCM. Prerna also created two new templates for Model Experimentation Rollout and Non-Model Experimentation
Rollout, which has helped in reducing high/low-sev tickets due to manual errors and provide better visibility to team and
stakeholders on rollouts.
TTA Migration – Prerna worked on Shifu to TTA service migration, used for distance calculation. As distance is one of the
primary inputs in pricing calculations, she migrated to TTA service in shadow mode and did a thorough analysis on Shifu and
TTA. Prerna worked with TTA tech team and RLB TPM to resolve the discrepancies due to API inputs, granularity and overrides
in Shifu/TTA for each region. She also proposed migration to addressId instead of zip-code based on deep-dive and sync-ups.
We were able to successfully migrate to TTA for all services in all regions without any major issues.
Peak Readiness – Prerna worked on 2021 prime peak readiness for one of the critical pricing services. She was able to identify
critical action items and documented the detail in radar doc, which was reviewed with team without any major callouts.
Prerna ensured to do load testing and ensured we are appropriately scaled for prime peak volume. We didn’t receive any
major high-sevs in 2021 prime peak.
Improving Developer Productivity – Prerna helped in multiple instances to improve developer productivity – she created SDC
review checklist and listed all the SOP for reviewing checklist which reduced churn and number of incoming tickets. She also
created new hire onboarding plan which details different topics/KT sessions in appropriate sequence, which has been used for
all new hires in the team and also created document to track sagemaker hosts counts for both inference/training
before/during/after deployment or rollout.

MENTORSHIP, DEVELOP OTHERS

Mentoring & Hiring – Prerna has been mentoring Samarth Singh, new SDE-Intern in our team (joined in Jan’2022). Prerna
Amazon Confidential v2022.04 5
LEVEL 2 – LEVEL 7 PROMOTION DOCUMENT
worked on her ramp-up plan, defined goals for his internship and ensured smooth on-boarding for him. She has helped him in
initial setup, understanding our technical system and about Relay from business perspective. She has assisted Samarth in few
initial tasks and also helped with his initial code-reviews. Samarth has been able to ramp-up quickly and has started
contributing independently. Prerna also helped Samarth in improving three growth areas – creating sub-tasks and better
scoping for the projects, better documentation for the suggested changes and improving coding quality. On hiring, Prerna has
completed MGHD & SDE-FI trainings and have completed 5 shadow and 3 reverse shadow interviewers and plan to become
independent interviewer by 7/30/22.
Helping Peers, Earn Trust -
Prerna has received 16 accolades in 2022 itself (highest in the team) and highest number of Sprint Champion awards in last
one year. She also received MMPT Champion Award in Q4’2021 as well. She presented new idea Friends and Family (FnF)
shopping experience in AITC’2021. Prerna along with Ajay Aneja took Learn a Tech session across Agrim-Team on Sagemaker,
which was well received by the attendees. She helped multiple peers in tickets, code-reviews and unblocking items like
running integration tests, reviewing MCM etc. She has also provided and influence design inputs in various situation like in
AMAP Metric Aware Guardrails (authored by Ajay Aneja, SDE-2 in our team), she suggested revisit our approach for calculating
guardrails hit, which finally became the recommended approach and similarly, in DAT Transit Time based Guardrails (authored
by Abhishek Saroha, SDE-2 in our team), she suggested and aligned on a simplified fallback approach to state code and transit
time band combination instead of relatively complex solution proposed earlier.

BEST REASONS NOT TO PROMOTE

At the time of promotion, it’s expected that an employee will have some areas for growth at the next level. This section should
be considered a positive exercise that will help identify those growth areas. These areas should be ones that you are confident
that the employee is capable of developing through coaching and on-the-job learning. Learn more about writing the best
reasons not to promote. (500 words or fewer are recommended)
Insist on High Standards – Prerna needs to work on balancing better between Bias for Action and Insist on high standards
(IHS). She some-times tends to bias for action and deliver results over IHS, which led to production issue once (as highlighted
in Abhishek’s feedback) and required few additional iterations before production release in couple of instances. Though, I
don’t see this as a blocker for her promotion, as there many counter data points in her code-reviews, design documentations
and deep-dives in on-call tickets etc.
Hire & Develop the Best – Prerna needs to improve on mentoring and hiring (as highlighted in Ayushi and Subhadeep’s
feedback). Based on her knowledge and skills, she needs to influence more inside and outside the team, specifically proactively
communicate the learning/new finding cross team (as highlighted in Ajay’s feedback). Prerna has also started taking interviews
but she hasn’t become independent interviewer yet partially because of lack of opportunities. I don’t see this as a blocker for
her promotion, but it would limit her ability to further grow and influence peers as an SDE-II.
Thing Big has been mentioned as growth area by couple of feedback providers (Subhadeep Gupta, Ruchit Shah). Though,
Prerna has shown multiple instances where she has been able to think big and deliver more than expected like in transit time
guardrails where she was able to develop solution for both Trips and Blocks (which wasn’t planned) and also identified existing
issues in transit time guardrails. She identified guardrails anomalies (new work stream) and worked with PM team to resolve
and did thorough benchmarking study for Anomaly Detection framework inside and outside Amazon. Thus, I don’t believe
Think Big as one of the “best reasons not to promote” for Prerna for L4 to L5 promotion.

WORK HISTORY (optional)

You can provide the candidates relevant work experience prior to this role here.
COMPANY POSITION PERIOD HELD
Amazon SDE Intern 13 Jan 2020 to 16 Jul 2020

ADDITIONAL INFORMATION (optional)

Supplemental information about the candidate: e.g., definition of terms, contributions to Bar Raiser Program, and/or
involvement in Amazon affinity groups. Specific organizational requirements: WWOps: narrative section on safety-focused
behaviors and/or contributions to inclusion & diversity initiatives. (500 words or fewer are recommended)

Amazon Confidential v2022.04 6

LEVEL 2 – LEVEL 7 PROMOTION DOCUMENT
The previous sections of this document (altogether) may not exceed 6 pages.

Overall promotion feedback are considered additional pages that are not counted in that total.
Copy/paste the table template below as needed for additional feedback providers.
FEEDBACK
FEEDBACK PROVIDER: Subhadeep Gupta BUSINESS TITLE & JOB LEVEL: Sr. SDE, L6
RELATIONSHIP TO CANDIDATE: Peer SDE FEEDBACK PROVIDER STEAM: Dave Clark
SUPPORT PROMOTION? Yes DATE OF FEEDBACK: 5/30/2022
Learn more about how to write feedback.
Best reasons to promote: (250 words or fewer are recommended)
In the last 2 years, Prerna has been able to independently drive multiple complex projects in RLB Pricing domain. Given a
problem, she is exceptional in breaking it down into multiple sub-parts and then propose a solution for each sub problem by
doing a detailed pros-cons analysis of multiple tech choices. For example, in the A-MAP trip transit time guardrails project, she
broke down the problem into raw data collection, data ingestion and processing, offline and online workflows. I have seen
similar approach in other projects as well. This approach makes sure that each part of the design has been given proper
attention, and hence the design review process also becomes easier for the rest of the team. For each project, she establishes
the motivation quite well by covering the current process and its limitations in extensive details (Sections 7 to 12 here, link).
She is good in figuring out multiple tech frameworks/solutions and then compare them exhaustively on multiple parameters
(link, link, link). She is very good in deep-diving issues. When research team reported an issue with Supply aware pricing input
dataset, she root-caused and unearthed a tricky code bug. She was the one who figured out the bug w.r.t always considering
price for lead-time 0, which could have caused major impact (high premium for Amazon) if undetected. Similarly, she actively
participates in the weekly pricing deep dive sessions, and has pointed out issues in pricing journeys of loads which others were
overlooking. While working on a task, she always goes above and beyond. For example, while working on the unified guardrails
project, she deep-dived into the existing business guardrails and figured out a lot of issues, which was not something that was
included in the requirements of her tech design. She worked patiently with product team and fixed those issues. While
working on the RLB Pricing launch in Canada project, she not only solved the Canada problem, but came up with a generic plan
and a checklist for different kinds of heuristic prices, so that launching any heuristic price in future becomes much simpler. Her
bias-for-action shines through when she is oncall, where she has consistently performed well to reduce the OE burden of the
team. She has also driven multiple OE and EE initiatives in the team, like addition of integration tests in Pricing service pipeline
to proactively catch issues prior to deployment, high severity ticket count reduction plan for the year, coming up with multiple
MCM templates and getting them reviewed by team-mates and also a MCM bar raiser to improve the overall MCM quality of
the team etc. For driving these initiatives, she has written exhaustive documentation to discuss all possible approaches
(link, link, link), and then discussed the same with the team. So, she has insisted on the highest standards in her deliverables.
She has been sprint champion multiple times, and has gotten numerous accolades from team members. She has established
herself as a seasoned campaigner in the team, and has earned trust from all team members.

Best reasons not to promote: (250 words or fewer are recommended)

Prerna can improve on the aspect of thinking big. In the vRLB (RLB price simulator) design, she could have considered the
future scope (w.r.t RL model and otherwise) in more detail, studied related designs, and then propose a more robust design
that also establishes the long-term vision. External (to the team) members found a little difficult to follow the design wiki
probably because the abstractions were not clearly mentioned. This caused some churn in the design discussions (like in
Samurai forum). This was the only instance where this happened though. So, this is not a promotion blocker, and I see this to
be more of an opportunity for her as and when she matures more into her role. She can participate in more hiring activities in
our org.

Amazon Confidential v2022.04 7

LEVEL 2 – LEVEL 7 PROMOTION DOCUMENT
Overall promotion feedback are considered additional pages that are not counted in that total.
Copy/paste the table template below as needed for additional feedback providers.
FEEDBACK
FEEDBACK PROVIDER: Ruchit Shah BUSINESS TITLE & JOB LEVEL: Sr. Manager - PM, L7
RELATIONSHIP TO CANDIDATE: PM Stakeholder FEEDBACK PROVIDER STEAM: Dave Clark
SUPPORT PROMOTION? Yes DATE OF FEEDBACK: 5/27/2022
Learn more about how to write feedback.
Best reasons to promote: (250 words or fewer are recommended)

Prerna is great at 1) Deliver Results, 2) Disagree and Commit, and 3) Earn Trust. I was Prerna’s product counterpart where I
worked with her on the launch of Marketplace Aware Search Space (MASS). Prerna earned trust by deconstructing all of the
product pricing metrics for me to help me understand how the tech capability overlaps with each of these metrics. In addition,
she pushed back and disagreed with me on our rollout strategy, ultimately aligning on a go-forward plan that was the best for
the business and the tech capability, which was iteratively tweaking our business pricing guardrails in parallel with the tech
rollout to help ensure our new pricing rules are baselined to the paradigm shift. We launched this feature in the US and IN,
which resulted in annualized entitlement of $135M and an estimated ’22 cost savings of $104M. From how she helped teach
the PMs how these product metrics overlap with tech capability, this led the PMs to pivot their strategy in terms of how we
optimize for cost (i.e., moving from floor guardrail hit rate to loads priced at lower bound). Not only did she exceed the Deliver
Results LP with the MASS rollout, but also developed the initial version of an RLB Simulator, which is a pivotal piece of
capability for the launch of RL-Based Pricing, and we’ll continue to scale and gain adoption with in ’23 and beyond.

Best reasons not to promote: (250 words or fewer are recommended)

Think Big. I view Prerna as one of my “go to” SDEs on the RLB pricing dev team. With her knowledge, I’d like to see Prerna
participate more in creating a long-term vision for the product. For example, how MASS can evolve with RL to offer more
automation and eliminate the manual overhead required by the product team to manage the business.

Amazon Confidential v2022.04 8

LEVEL 2 – LEVEL 7 PROMOTION DOCUMENT
Overall promotion feedback are considered additional pages that are not counted in that total.
Copy/paste the table template below as needed for additional feedback providers.
FEEDBACK
FEEDBACK PROVIDER: Ajay Aneja BUSINESS TITLE & JOB LEVEL: SDE-II, L5
RELATIONSHIP TO CANDIDATE: Peer SDE FEEDBACK PROVIDER STEAM: Dave Clark
SUPPORT PROMOTION? Yes DATE OF FEEDBACK: 5/24/2022
Learn more about how to write feedback.
Best reasons to promote: (250 words or fewer are recommended)

Deliver Results, Deep dive, Are Right a Lot/Team player: In her tenure within RLB Pricing team, Prerna has delivered multiple
ambiguous projects consistently which include Virtual RLB Simulator, new benchmark rate addition based on RCM booked
contracts, Market Aware Search space. All of these projects required thorough deep dive and data analysis. She also had to get
alignments from multiple stakeholders including research team, product team and RLB operations. Her works for reducing
guard rail anomaly have helped product team to launch MASS and save $ 135 Million. She also synced with research team
multiple times to define the VRLB success metrics and is constantly working with stakeholders to improve the accuracy. She is
an excellent team player and collaborate with multiple people to make sure here products are delivered on time, for example,
she worked with product team to make sure we are able to expand RLB in CA on time. She worked with multiple stakeholders
and made sure we are able to successfully test the features. Her work on coming up with pseudo hands on session for the LAT
sagemaker is also a great example of being a good team player.
Pricing anomaly detection design is a great example of deep dive, thinking big where she was able to evaluate multiple
platforms for doing anomaly detection and coming up with the design which will help pricing team catch anomalies faster. Her
design caters to both pre facto and post facto anomaly detection techniques. vRLB Simulator design also shows here ability to
come up with multiple approaches like choosing which environment to use, choice of the orchestrator etc and also how can
we roll out in a phased manner.

OE/EE, Insist on highest standards/Bias for action: She is also a front leader while improving team’s Operational load and
making sure we follow best EE practices. Her work to improve the Integration test suite and making them maintainable in
phase wise manner is one such example. She also led the effort to merge duplicate alarms for Pricing systems last year, this
has helped team reduce duplicate tickets. She also created two MCM templates for model and non-model experiments which
will help us standardize the experiment launch process and reduce the issues post launch. She is also actively mentoring a
new SDE intern in the team who is constantly delivering multiple items throughout his work till now. This shows great
mentoring skills as well. Her code quality has been consistently high wherein she covers all the branches with Unit tests for
example, in this CR wherein we added Date Object support in the ABExperimentsLib she handled all the unit test cases as well
as maintained backward compatibility. In this CR as well while adding RCM rates support, she maintained good standards and
required minimal updates. She is also one of the front leading code reviewer in the team and always strive for increasing the
code quality in the team: for instance in this CR she insisted in doing proper logging, completing documentation, following
good naming convention and made sure all branches are covered by UTs. She also got the Pricing Deployment MCM template
reviewed with the bar raiser and shared learning with the org.

Best reasons not to promote: (250 words or fewer are recommended)

Prerna should focus more on increasing her scope of influence beyond her team to other teams in the CDPT. She should start
working on increasing the visibility in CDPT Delhi. She should actively start mentoring people outside her team and take up
some cross team projects which can help her with visibility. As a tenured SDE I believe she can help the CDPT org to excel in
OE/EE by just proactively sharing the learnings/new findings across agrim-team. For instance, she shared the learnings about
MCM bar raiser and template arguments upon asking by her manager. We didn’t share the addition of Date Object support on
ABExperimentsLib Interest that might help other teams to use Dates in the conditions.

Amazon Confidential v2022.04 9

LEVEL 2 – LEVEL 7 PROMOTION DOCUMENT
Overall promotion feedback are considered additional pages that are not counted in that total.
Copy/paste the table template below as needed for additional feedback providers.
FEEDBACK
FEEDBACK PROVIDER: Ayushi Mittal BUSINESS TITLE & JOB LEVEL: SDE-II, L5
RELATIONSHIP TO CANDIDATE: Peer SDE FEEDBACK PROVIDER STEAM: Dave Clark
SUPPORT PROMOTION? Yes DATE OF FEEDBACK: 5/25/2022
Learn more about how to write feedback.
Best reasons to promote: (250 words or fewer are recommended)

Prerna has been a go-to person for any clarification around multiple domains for me. She quickly ramped up on Pricing and
delivered Canada Pricing. She demonstrated remarkable perseverance and ownership in deliberating MASS where she
consistently worked on detecting anomalies across Guardrails and working with Product to resolve them. It improved our
premiums and conversions and further streamlined the process of feeding guardrails to the model and applying above the
overridden price. Her data deep dive around unified guardrails, transit time guardrails, and tickets illustrates her excellent
deep dive skills. She has great knowledge of all the pricing systems. She also worked on introducing RCM based internal
benchmark for transit-time guardrails, for which she designed, followed up with Product, and thoroughly validated the RCM
data and did the implementation. With vRLB, she demonstrated her ability to work autonomously from end to end on a
project, leading the design, closing it out with many stakeholders and forums - team, Samurai, Research, and PMs – and
delivering it on schedule.
Prerna also excels in OE initiatives like as Integration Tests using Lugia, which helped to decrease churn around manual testing
before and after Prod deployment for OPS, saving us from numerous tickets, also worked on merging redundant alarms,
further lowering ticket count. She has become a trend setter by introducing BR reviewed MCM template for service
deployment, pricing model and non-model experimentation with exhaustive steps which can be followed up for clean
execution. These have helped a lot around setting up new experiments and executing steps in sequence as required.

Prerna has proved her commitment to high standards by writing high-quality modular code, making the components reusable
wherever possible. For instance, here https://code.amazon.com/reviews/CR-64050194 she used Factory Pattern to
incorporate any other incoming Output Transformer in future. She has also given meaningful suggestions improving the
documentation, coding style and performance evident here[https://code.amazon.com/reviews/CR-57508516] and
here[https://code.amazon.com/reviews/CR-56738208].

Prerna is also capable of breaking down ambiguous problems into well-defined smaller challenges which she then lays out in
detail, brainstorm, and comes up with a well approached out solution and plan of action like she did with the Guardrail
Anomaly Design. She first extracted out the requirements, obtained alignment on them from Tech, PM, and research and then
moved into deep diving into multiple existing systems solving similar problems. While presenting, she reinforced her points
with pros and drawbacks and kept the conversation on course.

She is always willing to assist peers, whether it is assisting me in setting up a new shadow experiment or pulling various
modeltype experiment results into a single sheet from the Alchemy dashboard.

Best reasons not to promote: (250 words or fewer are recommended)

Prerna should start taking more interviews, she is not active in hiring yet. That will build up her skills in interacting and
understanding others’ point of view even better. Prerna needs to better manage her time so that she doesn't stretch too much
and feel burnt out during tasks delivery, this is also not sustainable in long run.

Amazon Confidential v2022.04 10

LEVEL 2 – LEVEL 7 PROMOTION DOCUMENT
Overall promotion feedback are considered additional pages that are not counted in that total.
Copy/paste the table template below as needed for additional feedback providers.
FEEDBACK
FEEDBACK PROVIDER: Abhishek Saroha BUSINESS TITLE & JOB LEVEL: SDE-II, L5
RELATIONSHIP TO CANDIDATE: Peer SDE FEEDBACK PROVIDER STEAM: Dave Clark
SUPPORT PROMOTION? Yes DATE OF FEEDBACK: 5/24/2022
Learn more about how to write feedback.
Best reasons to promote: (250 words or fewer are recommended)

Prerna has delivered multiple complex items for CPAM successfully to prod including MASS, Trips Model Migration from
XGBoost to Random Forest, RLB Pricing Simulator, RLB launch in Canada, Trips and Block Guardrails based on RCM. She’s also
very consistent with delivering projects on time and able to do constant follow ups with product and research for any blockers.
She have shown great deep dive skills in providing inputs for previous price launch with unified guardrails. I worked closely
with her during launch of MASS(unified guardrails) with previous price. There were multiple conflicts and complexities while
launching these two experiments combined. She showed great deep dive and bias for action in making sure that both projects
are able to work with minimal issues. While working on MASS, she came up with multiple guardrail anomalies and did constant
follow-up with products, insisting on highest standards. She also worked on Trips and Block guardrails based on RCM, and
have good knowledge around guardrails and their anomalies and have become SME for the same. RLB pricing simulator also
had multiple ambiguities related to the request creation and carrier acceptance. She followed up with research and product
and came up with multiple approaches. She worked on MCM template, came up with different templates for model and non-
model experiments which is helping reducing the operational churn as well as tickets and helping in insisting on highest
standards, thereby increasing overall team productivity. Adding to this she also cleaned up our main MCM template which is
used for releasing critical services in production and also got it reviewed by Bar raiser. She also drived initiative of segregating
alarms and sanitizing the monitors which helped us greatly in reducing the number of tickets and also helped us focus on
actual issues with great confidence. During her on-call as well, most of the time she reduces the number of overall tickets
along with completing multiple actions items, there-by reducing the on-call burden for subsequent on-calls.
She has always been proactive in doing CR’s and giving valuable feedback on the CR’s. Prerna also helped on-boarding new
intern which helped him to deliver projects like DAT CA within months of joining in. She keeps raising the bar in code quality
and project deliveries. The work done by her during her tenure is commendable and she is all set to go to the next level.

Best reasons not to promote: (250 words or fewer are recommended)

Though Prerna shows high bias for action, this sometime leads to some issues which conflicts with insist on highest standards.
There was one instance while launching previous price with unified in which we fixed the issue related to converting pandas’
series to data-frame, but missed similar issue (converting pandas’ series having one row to data-frame) for other cases which
were failing more often. She needs to work on balancing bias for action with insist on highest standards and find the right
balance which can sustain both principles

Amazon Confidential v2022.04 11

LEVEL 2 – LEVEL 7 PROMOTION DOCUMENT

Overall promotion feedback are considered additional pages that are not counted in that total.
Copy/paste the table template below as needed for additional feedback providers.
FEEDBACK
FEEDBACK PROVIDER: Mohsen Moarefdoost BUSINESS TITLE & JOB Sr. Applied Scientist, L6
LEVEL:
RELATIONSHIP TO CANDIDATE: Research Partner FEEDBACK PROVIDER
Dave Clark
STEAM:
SUPPORT PROMOTION? Yes DATE OF FEEDBACK: 5/18/2022
Learn more about how to write feedback.
Best reasons to promote: (250 words or fewer are recommended)

Prerna joined Amazon RLB pricing team as SDE-1 in Jan 21, and we collaborated on several RLB projects, MASS (unified
guardrails), RLB Pricing Simulator (vRLB) and XGBoost for tours. In MASS, Prerna’s attention to details was impressive. She
continuedly conducted deep dives, identified and resolved root causes to deliver a high-quality software. In vRLB, Prerna
showed an exceptional ability to invent and simplify in order to improve runtime performance, she proposed the right
methodology to parallelize the computation and remove redundancies. During migrating Trips’ machine learning model from
Random Forrest to XGBoost, Prerna worked on almost all components/services and raised bar in ownership, insist on highest
standard. She provided positive and constructive feedbacks on ML model implementation and feature engineering through
code review. Also, she has been able to deep-dive on new initiatives and always comes up various approaches with detailed
pros/cons, which is reflected in various design/tech documents. Apart from this, she has drove multiple launch initiatives
deep-dive sessions with product/research team (guardrails anomalies, cost deviations).

Best reasons not to promote: (250 words or fewer are recommended)

Although I’m strongly supporting Prerna for promotion, there is a growth opportunity for her to learn and be curious, and
stretch her knowledge outside of her area of expertise.

Amazon Confidential v2022.04 12

LEVEL 2 – LEVEL 7 PROMOTION DOCUMENT
Overall promotion feedback are considered additional pages that are not counted in that total.
Copy/paste the table template below as needed for additional feedback providers.
FEEDBACK
FEEDBACK PROVIDER: Sudheer BUSINESS TITLE & JOB Principal PM, L7
Pamidighantam LEVEL:
RELATIONSHIP TO CANDIDATE: Product Partner FEEDBACK PROVIDER
Dave Clark
STEAM:
SUPPORT PROMOTION? Yes DATE OF FEEDBACK: 5/24/2022
Learn more about how to write feedback.
Best reasons to promote: (250 words or fewer are recommended)

Prerna raises the bar on Insist on high standards for L5 SDE-2 role. As part of working for MASS (Unified Guardrails), she
identified pricing anomalies that prevented Relay Load Board from pricing optimally. She collaborated with me and rest of the
product team to diving deep to identify improvement in pricing configuration and rates. Despite being new to the Relay Load
Board pricing team and to transportation pricing as a domain, she drove the initiative to address the improvement
opportunities by 1) Enabled a scalable solution to track and resolve pricing anomalies, and 2) setting up mechanism of alarms
and daily email alerts to all stakeholders highlighting these anomalies. She also raises the bar in ownership and deliver results
LPs by delivering complex projects like Model migration to Random Forest, Pricing Simulator and RCM-based guardrails.

Best reasons not to promote: (250 words or fewer are recommended)

Prerna’s development area is ‘Are Right a Lot’. To deliver results in a complex area like Transportation Spot Marketplace
pricing, encourage Prerna to proactively spend time in understanding the transportation marketplace dynamics from Product
managers and Operations’ SMEs before design. Examples: Understanding Box truck vs 53’ and need to combine different ‘per
hour’ and ‘per mile’ pricing would help her address the nuances in respective carrier segment needs and expectations

Amazon Confidential v2022.04 13