Download as pdf or txt
Download as pdf or txt
You are on page 1of 102

20230530- Required-Knowledge

It is imperative that all the content of this presentation is understood and at the end of the meeting
there are no doubts. This document is the merger of several of which you may already be aware.

We will go step by step so that your training is 200%.

If at any time you have a question, ask me.

At the end of the meeting, I will send you the document and upload in drive. It is recommended that
you review it by yourself and if you have any doubts and/or detect any inconsistencies, please let me
know as soon as possible.

If you consider it necessary to repeat the meeting to explain the contents of the document, no
problem.

Until you acquire full knowledge of the functional requirements, the project will not be able to move
forward.

Once you have mastered all the concepts and functionalities you can be the link with the technology
area. And you'll have more than enough autonomy to steer your project back on track.
Ready?
20230530- Required-Knowledge
• Radius Action Users (User = Radar)
• Ranges Ads (Intensity Signal Radio)
• Stablished in Extended Ranges Family and/or in Offers & Demands
• Calculating Distances
• Scope Users
• How to calculate Scope Users
• How antiscrape changes aplication behaviour
• Page Ads List vs Page Detail
• Ads List Prioritizing in Page Ads List
• Simulation Ads List ( version Equitable (Fair) & version Intrusive (Agresive)
• Flow & Activity users
• Preserving Channels to be contacted
• Anti Scrape measures
• Sensitive Data & Fake Data
• & Edit Phone Numbers
• & Create , Update User
• Managing Email message service when antiscrape has beed activated
• User Data Management

It seems like a lot of content but it is necessary and everything is related.


We use the word Category to agroup Families & Subfamilies

Radius and Ranges are in fact the same concepts: They are Scopes

• Radius is used for users


• Ranges is used for Ads

Aplication manage Latitud & Longitud geographic coordinates of:

• Users (Visitants or Registered) that are Geolocated in base of IP


• Ads that are Geoubicated when owner create Ad

You must have in mind that Mega Ads:

will allow to create Ads about Services and Products that can be new or used:

The goal of MegaAds

Avoid displaying expired ads, for this reason owners must renew Ads.

There is nothing worse than contacting an advertiser and they either ignore you or they no longer have what they advertised.

Show users ads that are not relevant to their intentions.

Does it make sense for a user to see an advertisement for a second-hand product 200 km away? .

Does it make sense for a user to see an advertisement for a service provided 200 km away?
Radar
Managed with the Radius Action Users

Range Ads
As the intesity signal radio
Managed with
• Extended Range Categories
• Offers & Demands
We use Radius Action Users to define the visual scope of users in Km

Registered Users will have a greater Radius Action than Visitant The best way to understand fully the concepts of Radius
is to use a metaphor
Radius Action Users is managed in
Aplication must treat the user like if he was a radar and use his
Manager > Pub Features > Radius Range > Radius Action Users
gelocation to find ads inside his radius action

There is the user


And his radius action in KM

Ads geopositionated inside


Radius Action user will be
displayed by aplication to user
from any family or subfamily.

When Anti Scrape is active, it can automatically reduce the Radius Action USers
Stablished in Extended Ranges Family and/or in Offers & Demands
We use Ranges to define the visual scope of Ads in Km
Range Ads is managed in
Extended Family The best way to understand fully the concepts of Ranges
is to use a metaphor
Manager > Pub Features > Radius Range > Radius Range Subfamily Extended

Aplication must treat the Ads like if they was radio stations with
different degree of intensity of a radio signal. And in base of
the Ad geoposition and his intensity, the Ad will be visible or
not for the user.

The intensity of the Ad will be stablished by:


• Radius Range Subfamily Extended
Staff can Extend Range of families and/or Subfamilies • Offer & Demands

Offer & Demands Radius Range


Manager > Pub Features > Radius Range > Offer Demand Radius Range Ad with no intensity

Ad with low intensity

Ad with medium intensity

Ad with high intensity


The contributed values are not very descriptive
Calculating distances between user and Ads using latitude and longitude
latitude and longitude
Geolocation User 40.44438 , -3.70199

17,92 km

latitude and longitude


Geoposition Ad 40.46842 , -3.49256

The application must not use third party resources for distance calculation.

Haversine's formula must be used to calculate distances


Let see how will
work aplication
if we only use
• Radius Action Users
Let see with examples No category with range extended, so Ads visible for user will
We have 3 fam or subfam categories be ads geopositionated inside his radius action …ok?
Name Dot Range Extended
Cat01 Green 0 Km The signal strength of the advertisement does not matter, as
Cat02 Orange 0 Km long as it is within the radius of action user. Ok?
Cat03 Red 0 Km

9
Radius Action Users
for Registers Users

2
Visitant users will see ads: 1 & 5
6 Registered users will see ads: 1, 5, 2, 3 & 4
Ok?
1

5
3

7
Radius Action Users
for Visitants Users 4

8
Let see how will
work aplication
if we use
• Radius Action Users
• Extended Range Categories
(Families or Subfamilies)
Let see with examples
We have 3 fam or subfam categories
Category Cat08 has extended Range to 100 Km
Name Dot Range Extended
Cat01 Green 0 Km
Cat02 Orange 0 Km
Cat03 Red 100 Km

9
Radius Action Users
for Registers Users

2
Visitant users will see ads: 1 & 5
6 Registered users will see ads: 1, 5, 2, 3 , 4 & 9
Ok?
1
Ads: 1, 5, 2, 3 & 4 will be count as LRPD-Inside
5 Ad 9 will be count as LRPD-Outside
3 Ok?

7
Radius Action Users
for Visitants Users 4

8
Let see how will
work aplication
if we use
• Radius Action Users
• Offers & Demands
No Extended Ranges in Categories ( Fam or Subfam )
Name Product Service Range Extended
Cat01 Red Green 0 Km
And geoposition Ad will respresented by colors
Cat02 Red Green 0 Km depending if is a Product )Red Dot) or a Service
Cat03 Red Green 0 Km (Green Dot)
Let see with examples using Configuration Offers & Demands
Visitant users will see ads: 1, 5 & 6!
Registered users will see ads: 1, 5, 3, 2!, 4. 6! & 9
Ok?

Ads: 1, 5, 3 & 4 will be count as LRPD-Inside


100 100 100 100 10 10 10 10 Ads 2, 6 & 9 will be count as LRPD-Outside
Ok?
Suppose a site with only Offers, and for the Example we are not diferentiate ranges in Local
or Shipments / Displacement and we dont diferentiate if ad are free or Paid. 9
The size of the lines and/or radius
2 are not proportional but it is useful
to understand how it works.

5
3
7

4 6
8
Radius Action Users Families Radius Range Extended
Can work as Default or Activated Control Radius
If selected we say ON, if ON aplication must use values of Extended Ranges

Offers & Demands


If selected we say ON, if ON aplication must use values Ranges of Offers &
Demands for Products (Local & Shipments) and Services (Local &
Displacement) depending Free or Paid Ads

I have hide all values in order to not


generate confusión.
How work Radius , Extended and Offers & Demands?
Radius Action Users Families Radius Range Offers & Demands
Extended

Staff will stablish Radius Actions Users for Staff can allow to Families & Subfamilies to Staff can stablish differents ranges for Ads
visitants and Registered Users. have Extended Ranges that will affect to Ads depending if Ads are Offers or Demands,
associated to family or subfamily with the Products vs Services, Local or
Option Default is the normal option BUT range Extended. Shipment/displacement , Free vs Paid.
Staff can Select Manually any time to Active
Control Radius and reduce the radius actions Important, In order aplication use Extended Staff can manully select option to active that
users inmediately. Ranges in Families or Subfamilies, the aplication will use values or unselect option.
checkbox must be selected.
It can also the default option be reselect
manually by Staff and allow to users biggers Once the list of Fam & Subfam with Extended
ranges. Ranges are created. Staff can manually
deactivate to use Extended Ranges by
unselection option. Or to reselect option to
active Extended Ranges without redoing ther
list of Families or Subfamilies.

So, aplication can work using always Radius and dependin if options are selected, aplication can work in 4 ways:

• Using only Radius Action Users


• Using Radius Action Users plus Ranges Extended
• Using Radius Action Users plus Ranges Offers & Demands
• Using Radius Action Users plus Ranges Extended plus Ranges Offers & Demands
How work Anti Scrape? Wich options will atomatically change in case of Anti Scrape is activated and triggered?

Radius Action Users Families Radius Range Offers & Demands


Extended
A ON / OFF ON / OFF
B

When Antiscrape is triggered, aplication will When Antiscrape is triggered, aplication will When Antiscrape is triggered, aplication will
check option selected in Radius Action Users. check option selected in Extended Ranges. check option selected in Offers & Demands.

Atention here.
if it was already selected by Staff the option if it was already unselected by Staff
(B) , aplication will leave it as it is. aplication will leave it as it is. … current site is If it was already unselected , aplication will
not using Extended ranges in family or select it.
If option selected was Default (A) subfamily.
Aplication will change option and Select to
Activate Control Radius. (B) If it was selected by Staff (current site is using If it was already selected, aplication will
This will allow to reduce Radius Action Extended Ranges Families or Subfamilies) leave as it. Aplication is using ranges for
Users. …aplication will change selection… so, Offers & Demands.
aplication will unselect Extended Ranges.
This will allow to cancel Extended Ranges in
Families and Subfamilies.
First revised countermeasure action Second revised countermeasure action Third revised countermeasure action

Radius = B If OFF then nothing. IF ON then nothing.


If ON then OFF IF OFF then ON

So, Anti Scrape activation can change radically the scope of users and Ads!
You must have in mind that aplication must work depending on configuration stablished by Admins.
In tables i use F1 & F2 as categories (Fam & Subfam) and P & S for Product and Services.
Radius action Users same for Registered than visitants. Default Radius 100 km and Active Control Radius 50 Km

Each case represents diferents configurations site with the scope in normal situation (left Side) and in right side when antiscrape is activated the new scope

Normal Situation Anti Scrape Activated


Radius Action Users Extended Ranges Offer & Demands Ranges Radius Action Users Extended Ranges Offer & Demands Ranges
Case 1

(A) Default 100km OFF OFF , P = 1 km , S = 1 km (B) Activated control OFF ON, P = 1 km , S = 1 km P = 51
to 50 km S = 51
Max Scope will be 100 km for any cat and Ads products & Services
Max Scope will be 51 km for any cat and Ads products & Services

Radius Action Users Extended Ranges Offer & Demands Ranges Radius Action Users Extended Ranges Offer & Demands Ranges
Case 2

(A) Default 100km ON (F1=0,F2=300 km) OFF , P = 400 km , S = 20 km (B) Activated Control OFF (F1=0,F2=300 km) ON , P = 400 km , S = 20 km P = 550
to 50 km S = 70
Max Scope will be 400 km for Products & Services in Cat F2. Max Scope will be 450 km for Products in any Cat.
… but for Cat F1 max scope will be 100 km for Products & Services … but for Services, max scope will be 70 km for any Cat

Radius Action Users Extended Ranges Offer & Demands Ranges Radius Action Users Extended Ranges Offer & Demands Ranges 450 Km
Case 3

(A) Default 100km ON (F1=0,F2=300 km) ON, P = 400 km, S = 400 km (B) Activated Control OFF (F1=0,F2=300 ON, P = 400 km, S = 400 km P = 450
to 50 km km) S = 450
Max Scope will be 800 km for Products & Services in Cat F2. Max Scope will be 450 km for Products & Services in any Cat.
… but for Cat F1 max scope will be 500 km for Products & Services

Radius Action Users Extended Ranges Offer & Demands Ranges Radius Action Users Extended Ranges Offer & Demands Ranges
Case 4

(A) Default 100km OFF (F1=0, F2=300) ON, P = 400 km, S = 50 km (B) Activated Control OFF (F1=0, F2=300) ON , P = 400 km, S = 50 km P = 450
to 50 km S = 100
Max Scope will be 500 km for Products in any Cat.. Max Scope will be 450 km for Products in any Cat..
… but for max scope will be 150 km for Services … but for max scope will be 100 km for Services
Page

ds Detail
Page Ad List vs Page Detail Ad

Pages Ad List are the Pages where Ads are listed as a claim
and where user need to click to load info Detailed Ad.

1.-Page Ad list 2.- Page Detail Ad Pages Detail Ad is the Ad Page with the Title,
Subtitle, Description , Pictures, Documents. …
Lorem ipsum dolor
and the most important: Channels to contact with
Aenean efficitur
suscipit dui sit amet owner ad.
ultrices

Lorem ipsum dolor sit


amet, consectetur
adipiscing elit.
Maecenas eu laoreet
magna. Proin
scelerisque euismod
orci. Aenean efficitur
suscipit dui sit amet
ultrices.
Channels to contact with owner ad.

3.- The sensitive data info is the data to contact


with owmer ad as email or Phone number.
Ads List Prioritizing
in Page Ads List
Page Ad List
To build Page Ads List by default or when user do a search , aplication must first:

• Locate Ads available in DB based on Radius Actions User, Extended Range Categories and Offers & Demands

Once Ads available has been located in DB, aplication must organize and order the ads to display based on:

• Properties of the ads will prioritize current than non current, Paid than Free Ads, certified than non certified

• Paid Ad current certificated


• Paid Ad current non certificated
• Free Ad current certificated
• Free Ad current non certificated

• Paid Ad non current certificated


• Paid Ad non current non certificated
• Free Ad non current certificated
• Free Ad non current non certificated

• Each of the 8 groups above will be sorted by:

• Timestamp Last Event from the most Most recently updated (repositioned and/or renovated) to the least recent
• Note: for ads non current timestamp will be always last daily renewal because non current can not be repositionated, ok?
Simulation Ads List
Reposition Ads
Equitable (Fair)
Intrusive (agresive)
Remember

Planner daemon has:


• 2 Daily Renewal hours, one for Free ads and the other one for Paied ads
• Several hours stablished for Payed Ads plan in order owner ad can mark to do automatically repositions ads
• Several cycle for free ads that limit the number of times that owner can reposition his ad in same day

And, owner Paid ads can do repositions with no limit, each time they want if they have tickets, they can do.

Each time that Daemon run , it will update timestamp Last Event … that can be for
• Automatic Renewal Daily Ad
• Automatic Repositions Paid Ads

And between the hours of Daemon executions has be planned, users can be active and do his own repositions ,
• Manual activity from users must be reflected on Last Event Timestamp Ad
BASIC FLOW that include
• Current
• Non current
Dameon has run two times: Query at 09:00 am
• At 07 am to renew Free Ads Dameon has updated Last Event with timestamp execution
• At 08 am to renew Paid Ads

Daemon process must update timestamp Ads Last Event taking into
account how ads must be sorted / displayed to users prioritizing :

• Current than non current


• Certificated than non certificated
• Paid than Free Ads

Once Daemon process has done his task, Aplication can do Select same
as prioritizing and add to display sort list ads using timestamp from most
recent to oldest.

Lets see… Ad non current


• Paid first than free Ads
• Certificated first than non certificated
Dameon has run two times: Query at 09:00 am
• At 07 am to renew Free Ads Dameon has updated Last Event with timestamp execution
• At 08 am to renew Paid Ads

Ad non current
• Paid first than free Ads
• Certificated first than non certificated
Some of them have been repositionated between: Query at 13:00 pm
• Last execution Daemon at 08:00 am
• Query at 13:00
More have been repositionated Query at 17:00 pm
Version
• Equitable
(non agresive more fair)
Detail flow with:
Daemon process updates prioritizing
• Current , Certif , Modal
Aplication display list
• Current , Certif , Modal, timestamp
Query done at 08:02 am

Certificated
Paid At 08 am to Daily Renewal
Group sorted by
most recent to oldest
Daemon process must update timestamp Ads Last Event taking into
account how ads must be sorted / displayed to users prioritizing :

• Current than non current

Certificated
• Certificated than non certificated
Group sorted by

Non
• Paid than Free Ads most recent to oldest

Once Daemon process has done his task, Aplication can do Select same
as prioritizing and to display sort list ads using timestamp from most
Free At 07 am to Daily Renewal

recent to oldest.
Certificated Group sorted by
most recent to oldest
Lets see…
Certificated

Group sorted by
Non

most recent to oldest


Query done at 08:02 am

Certificated
Paid At 08 am to Daily Renewal
Group sorted by
most recent to oldest

Certificated
Group sorted by

Non
most recent to oldest
Free At 07 am to Daily Renewal

Certificated Group sorted by


most recent to oldest
Certificated

Group sorted by
Non

most recent to oldest


Ads 11, 28 ,7 & 5 has win positions… Ads 27, 23 & 14 has win positions…
Look timestamp from automatic proccess

Query done at 08:02 am Query done at 09:35 am Query done at 10:01 am


Owner from Ads 11,28, 7 & 5 will manually Daemon at 10:00 will run a Hourly Relocation to
do a reposition between 9:00 am & 9:35 am automatically reposition ads planned: 14, 23 & 27
at differents moments. In this case, as free ad 5 has been listed before than
Paid Ads it can not be cosiderated Equitable.
Ads 27, 23 & 14 has win positions…
Look timestamp from automatic proccess

Query done at 10:01 am Query done at 11:55 am


After Hourly Relocation has run at 10:00 am, VERY IMPORTANT
users will continue doing manually repositions Version Equitable
of ads: 28, 9, 18, 17, 8, 6 & 21 (non agresive, more fair)

In the Equitable (fair) version, even if


free ads has been just repositioned,
they will never be shown before paid
ads …. ¿reason? Prioritization!.

The query at 11:55 am…

Free ad 17 repositionated manually at


11:18:33 have a timestamp that is most
recent than Paid ads:
14, 7, 28, 27 & 23
repostionated between
9:22:08 and 10:00:01

This version is really fair for onwer


Payed Ads.
Version
• Intrusive (agresive)
Once versión Equitable will be implemented, we will request Intrusive that is more simple to developpe

Detail flow with:


Daemon process updates prioritizing
• Current , Certif , Modal
Aplication display list
• Current , timestamp
Daemon process must update timestamp Ads Last Event taking into
account how ads must be sorted / displayed to users prioritizing :

• Current than non current


• Certificated than non certificated
• Paid than Free Ads

Once Daemon process has done his task, Aplication can do Select but
this time the prioritizing …will only be do:

• Current than non current

And using timestamp from most recent to oldest.

Lets see…
Query done at 08:02 am

Paid At 08 am to Daily Renewal

Certificated
Group sorted by
most recent to oldest

Certificated
Group sorted by
most recent to oldest
Non
Free At 07 am to Daily Renewal

Certificated

Group sorted by
most recent to oldest
Certificated

Group sorted by
Non

most recent to oldest


Query done at 08:02 am Query done at 11:55 am
VERY IMPORTANT
Version Intrusive
(agresive, less fair)

In the Intrusive (less fair) version,


aplication will disply list ads just doing
priorization to differentiate Current than
non current but sort will be done using
timestamp from most recent to oldest.

The diference is clear, free ads manually


Automatically
repositionated
repositionated can be displayed before
Paid Ads manually repositionated or
automatically respositonated using
hours in plan.

This version is less fair for onwer Payed


Ads, specially if they dont plan Hours to
be repositionated.

If owner payed ads want to have always


preferably listed the first ones … not
only they will need to Select hours to
automatically respositionated his ads
but also is posible that they will need to
do it manually too.

Intrusive versión rewards the ads that


are most atended by owners.
Query done at 08:02 am Query done at 11:55 am Query done at 17:30 pm

Automatically
repositionated
Admin / New option [ Reposition Ads ]

In Admin > Config Public Site …. As last option, add “Reposition Ads”

Not do do now.
Once Equitable versión will be
done and tested …

I will request improvement


versión Intrusive

Radio button
Flow & Activity users
User activity for visitants and/or registered users
When user access to the Public Site
The application must obtain the Geolocation using the user Ip.

Once aplication know Geolocation user, aplication knows the coordinates of user (latitude and longitude
coordinates) and city.

And depending the values stablished by Staff in configuration


Admin > Config Public Site > Geolocation
It will use as a “preset” later

The application must take into account the values of:


• Radius Action Users &
• Offer & Demand Range Services & Products &
• Extended Range Family

In order to list Ads:


• by default or
• when user will do normal Search or Advanced Searh
(words typed in bar searh, category selected and/or using Filters)

In the meantime, the user will be able to surf in site loading pages and pages with ads listed, he can redo
searches as many time as he want, …but : Once user decide to look a Page Detail Ad

The aplication will check the Geolocation Preset to allow or deny to display Page Ad Detail (if is a Paid Ad
or Free Ad or Both!) or once Page Detail is loaded do not allow user to contact with owner Ad.

When aplication allow to load & display the Page Detail Ad to user…

Each time that user visit a Page Detail Ad, aplication will treat in his sesion user: To count correctely when the user is looking Page Deatil
• Count number of Page Detail Ad consulted & aplication must manage distance to identify
• About Page Detail Ad … • Inside Radius Action Users (LRPD-Inside)
• Count number Ad Inside his Radius Action User (LRPD-Inside) • Outside Radius Action Users (LRPD-Outise)
• Count number Ad Outside his Radius Action User (LRPD-Outside) Ads Outside Radius Action Users but “inside” the Range Ads

Activity session user will increase the values of LRPD-Inside & LRPD-Outside in order to check if Antiscrape
is active or not. In case that antiscrape is active, aplication will compare limits LRPD or/and TMRPS (Total
Maximun Request Per Session) in order to react to a potential data mining process
Preserving
Channels to be contacted
Preserving Channels to be contacted

Chat: If the user use the chat to contact with owner, there are no risk for us (Company). If user or owner decide to
cross emails, phone numbers or any other data… it will not be our problem.

Email: If the user use the email to contact with owner, there is a risk for us (Company) if aplication display email
owner to user, or aplication use a link that open any email utility.

I suppose in order to preserve email owner this is done opening a form in order to request to user the text that he
want to send to owner ad and his email in order that owner ad can reply user using his own email and using his
own email service.

… as i have not the oportunity to display for the moment a Page Ad Detail …i have not checked feature.
¿How process to contact by email with owner email has done?

Phone number: With Phone Number user can contact owner Ad using Whatsapp or just calling him directely. If
owner ad has acepted our terms of conditions and is using a category where phone number is required or optional
and owner ad has decide to enable Phone Number.

Even if the owner ad has authorized the publication of his phone number, we must take measures to protect access
to his mobile number, or to the mobile phone numbers of ads on a massive scale using external process (bots, etc).

Actually with European Regulations we must be care sharing sensitive data. An we must take
measures to preserve sensitive data. The Anti Scrape will be our mesure with special treatments.
Once it’s clear what is:
• Page Ads List vs Page Detail
• Data sensitive from owner
• Radius users & Ranges Ads
let see how Works the Anti Scrape
Anti Scrape measures
Anti Scrape mesures

Have in mind that aplication is designed to work as a Generic Site or Specialized Site. Public Site has 2 templates
If you need more detail about the two templates, let me know.

So depending if site is Generic or Specialized and the type of categories (fam & Subfam) … data info to contact with owner must
be preserved to a greater or lesser degree.

So, the values to use in Radius Actions Users, Offer & Demands, Extended Range fam / Subfam will allow Staff to have site more
restrictive or more tolerant of user activity when requesting and/or viewing details of advertisements.

And on the other hand, we have the antiscrape that can be not active or active as a the guardian of the tower.

Anti Scrape mesures are required in order to:

• Provide erroneous alternative information instead of sensitive information


• Hinder automated external procedures
• Temporarily stop automated external processes
• Stop the extraction of sensitive data
• Stop comunication emails

Anti Scrape mesures must affect the user session who is performing them manually or automatically by means of bots, third party
tools, etc...

This means that all measures adopted in the user's session should not be applied to the rest of the users using the web.
Measures = option selected in Anti Scrape. In next slides I detail wich measures are only for user session and wich meauser
can affect to all the users in all the session users.

Let see how must work Anti Scrape mesures


Anti Scraping
AntiScrapes mesures will be determined in base of configuration options in Admin > Config Public Site > Anti Scraping.
Options are:
• Active Anti Scraping How works Anti Scraping options?
• Max Request Per Session
• TMRPS Management The 3 options are explain in nexts slides

Admin > Config Public Site > Anti Scraping > Active Anti Scraping Admin > Config Public Site > Anti Scraping > TMRPS Management

Admin > Config Public Site > Anti Scraping > Max Request Per Session
Anti Scraping /

Anti Scrape is sleeping 100%, (1) no selected is mean no Active


1 Anti Scrape can be activated as soon as Staff Select option (1) and almost one of the options (2) or (3)
2 A TMRPS is the sum LRPD-Inside & LRPD-Outside C=A+B
If only option (2) or (3) is selected , the TMRPS will be the value of the option selected. Ok?
3 B
Values A & B contributed in option “Activate Anti Scrape” must be displayed and can not be changed in
C
option “TMRPS Management”.
To allow aplication to react and do option (6) , options (1) & (2) must be selected
To allow aplication to react and do option (7) , options (1) & (3) must be selected

Admin > Config Public Site > Anti Scraping > TMRPS Management

4
A
5 6

B
7
If Anti Scrape is active, option (1) in option “Activate Anti Scraping”,
Staff can stablish when Total Request Per Session has been
surpassed by user …. what aplication will do: 8 K

(4) Send Email … as an alert to email informed in text area box.


(5) Enable one of the Captchas available

Atention: Email alert it will send just one time just when the TMRPS 9 L
has been surpassed,… if the TMRPS is stablished in 100, the email
will send once the TMRPS counter will have the value 101. Is user
continue to request/load Page Ad Details …the TMRPS will 10
M
continue increase counting, but aplication will no send more
emails. Only 1 for each user Session when TMRPS will be surpassed.
Ok?

Any or all cases can be selected by Staff users.


Anti Scraping /
Admin > Config Public Site > Anti Scraping > TMRPS Management

A
6

B
7

8 x 125
K

9 x 100
L

10 x 150
M

Once Anti Scrape has been selected to be active (1) , the number of Page Ads Details
requested by user in his session must be counted ( inside, outside & total)…

If Staff decide to Select Case C, D, and E … values contributed in K,L & M must be
greater than A & B …. And for the values of K,L & N ,,, Staff will be free to decide
which to use depending the priority of needed.

And in adition of the measures tan can be activated in TMRPS Management, do not
forget that Staff can or not to activated diferent captcha.
Anti Scraping /
Admin > Config Public Site > Anti Scraping > TMRPS Management Once Anti Scrape has been selected to be active (1) , the number of Page Ads Details
requested by user in his session must be counted ( nside, outside & total)…

If Staff decide to actívate the best option in Anti Scrape is to Select Case B.
A
6 This will do that aplication react with the user session and as soon the number of
Page Detail Ads requested is greater than the limit stablished LRPD-Outside ( B ) …
remember… Outside Radius Action user ….
It will activated to show Fake data information

7 x B
So, the bot or the user can be surfing in site and requesting Page Details Ads and
requesting Page Details Ads …. but aplication will show fake data information from
owner ads, … aplication will preserve original sensitive data from owner….
K
8 100
Please refer to document request proyect. If you have any doubt about fake data
info, ask me.

9 L
125

M Note: when show Fake data information will be activated, service message (email)
10
150
will not be sending messages to owner ad sended by current session user.
Anti Scraping /
Admin > Config Public Site > Anti Scraping > TMRPS Management Manager > Pub Features > Radius Range > Radius Action Users

A
6
100

150
B
7

10
K
8
15

9 x L

10
M Manager > Pub Features > Radius Range > Radius Action Users

100

Once Anti Scrape has been selected to be active (1) , the number of Page Ads Details
150
requested by user in his session must be counted ( nside, outside & total)…

If Case D is active (9) and the TMRPS has been superated, aplication will auto change
in Radius Action Users option from “Deafult” to “Activated Control Radius actions 10
users” …,

Staff will be need to rechange option selected in order to restore Radius Actions 15
Users Value.

Note: Value Contributed by Staff in Case D will be greater than the value obtained in
Admin > Config Public Site > Anti Scraping > Active Anti Scraping
Just with a few Ads and configuring Anti Scrape with low values, to test or check Anti
Scrape will be very easy.
Sensitive Data
& Fake Data
& Edit Phone Numbers
& Create , Update User
Data sensitive
We call data sensitive to personal data of owner ad, and MegaAds is designed to allow users to contact with owners ad using:

• Email owner
• Phone numbers owner
• Chat
Depending how Staff will stablish properties of families & subfamiliies, owner Ads can optionally or mandatory acept that users contact with owner ad
using channels to be contacted.

The chat is not a problem, it not require to protect any personal data sensitive from owner.

The email is not a problem too, if aplication do process without inform to user the email owner ads. Is mean, that is absolutiñy forbidden to use in
button email to owner with something so basic and obsolete in terms of best practices like a link href="mailto:emailownerad@gmail.com"

The phone numbers is a big problem… even if the owner ad has authorized users to contact him using mobile phone number, we must have to take
into account that it is publicly exposed information at the mercy of any user and/or company.

Our responsibility and obligation to the best of our ability (and there is no shortage of them) will be:

• Protect public data when session activity is unreasonable, i.e. when behavior is atypical.
• Do not make it easy for competing companies (or users with unknown intentions) that want to obtain data from third parties for their own use.
• And don't make it easy for automated processes either

To carry out our responsability and obligation, MegaAds is designed with:

• A control to list ads in user session based on:


• User geolocation witha a Radius Action (Radar) and
• Ad geopostiion Ranges (Intensity signal Ads) that is variable
• Antiscrape

Let’s make and imaginary and creative scenario…. We don’t need to use any value, just visual concept. But it will help you a lot to understand it.
Imaginary and creative scenario, suppose a square country full of ads…

If aplication do not control Geolocation user and


Geoposition Ad, the aplication will list all ads in
the country to users doing request using category
or doing searchs.

A automatizated process can obtain all data


sensitive from owner ads … even if a record was
kept of how many times in the session it has been
requested a Page Detail.

No matter how many times a control would be


triggered in the session when the number of
times it has been a Page Detail requested is
exceeded …. In a few days, all data can be obtain.
Imaginary and creative scenario, suppose a square country full of ads…
Green: Normal Range Ads
Red: Extended Range Ads
Radius Action User in blue
As soon as aplication manage the concepts of:
• Radius Action User and
• Range Ads

Most of the Ads will not be listed by aplication, if


will be as if Ads will not exist for user.
Imaginary and creative scenario, suppose a square country full of ads…
Aplication will list to user, Ads inside radius action user (Radar) and Ads Even if the same user decides to use a vpn to change their location, the
outside radius action user because Ads have an extended range (higher application will still be effective, in case of a process to scrape data.
intesity signal). Rest of ads in country…no exist for the user.
Antiscrape
In this example we have:
• 12 ads Inside Radius Action User (Radar)
• 10 ads Outside Radius Action User (Intensity Signal)

For a better understood suppose that each dot is equivalent to


10 dots! So in reality the user has access to see listed:

• 120 ads Inside Radius Action User (Radar)


• 100 ads Outside Radius Action User (Intensity Signal)

A total of 220 Ads at his reach.

Usually or normal activity from users when they are doin using
sites is not to see in the same session more than 50 Pages of
Ads Detail.

Although there are users who spend 100% of their time


searching and searching, they are a very, very low percentage
and we are not going to worry about them.

So, Staff stablish in AntiScrape values limit for:


• 10 ads inside Radius
• 10 ads outside Radius

The Total Maximun Request Per Session will be 20.

I use low values because is more easy for me to display it in next


slides.
The Total Maximun Request Per Session will be 20.

So, the first 20 Page Detail request by user will display to user
the real phone number owner Ad

If antiscrape is active, when the limit TMRPS is exceeded,


aplication will not display the real phone number owner!

Instead, it will display a mobile number whose last four digits


have changed randomly.
The Total Maximun Request Per Session will be 20.

So, the first 20 Page Detail request by user will display to user
the real phone number owner Ad

If antiscrape is active, when the limit TMRPS is exceeded,


aplication will not display the real phone number owner!

Real Data Phone Numbers


Instead, it will display a mobile number whose last four digits
have changed randomly.

This measure almost seems ideal but has a flaw....

Lets see next slide

Fake Data Phone Numbers


Although it is highly unlikely, let's assume that:
• ads are always the same
• they are always listed in the same order

The example represent the fake phone numbers displayed in


different scrape done probabily by same user a company.

Real Data Phone Numbers


Always aplication will display
Real Phone number from #1 to #20
Fake phone numbers from #21 to ….

Once the information was downloaded to a third party, a simple


comparative analysis of the data content would lead them to a
conclusion quickly….

Round 1 Round 2 Round 3 Round 4 Round n

Fake Data Phone Numbers


A simple comparative analysis of the content would give the
application away and confirm their suspicions.

The conclusion reached would be that variable information is


being suministrated by aplication and is therefore likely o surely
to be false / fake.

Real Data Phone Numbers


This conclusion will lead them to other alternatives that could be
more harmful to the application and/or the server...

So ...how false information should be generated


and treated?

Round 1 Round 2 Round 3 Round 4 Round n

Fake Data Phone Numbers


A very simple solution to to prevent the authors of the scrape
from perceiving that the information obtained is false
… is to always show the same false information.

To fulfill the objective, each time a user contributes and/or


modifies his phone numbers, the application must randomly
generate the fake number and store it in its corresponding fake

Real Data Phone Numbers


field.

So each time, that antiscrape will be triggered and aplication


will react… the fake Phone Numbers displayed will be fake, but
always it will be the same fake Phone numbers.

Round 1 Round 2 Round 3 Round 4 Round n

Fake Data Phone Numbers


Managing Email
Service Message
when antiscrape
has beed activated
The email Is absolutiñy forbidden to use in button email to owner with something so basic and obsolete in terms of best
practices like a link href="mailto:emailownerad@gmail.com"
Process emailing user from Page Ad Detail (when email is availabled in channels to be contacted )
1.- Registered User press button email in page detail Ad (not allowed for visitants)
2.- Aplication display Form in modal windows to message email to onwer ad
Note: Fields From & Subject are not editable, user can not update it.
From: Aplication will fill with the email of registered user that want to send message to owner ad
Subject: Will be the Tittle Ad ( Not editable)
Texarea: Where user will write his message
Button Cancel: will close form without doing nothing
Button Send: will continue process

3.- Internal aplication will request db email owner and send data form (If Antiscrape has not been triggered and Case B is selected)

4.- The registered user will receive a copy of email sended to owner Ad.

5.- A copy of email sended to owner Ad will be send too to email Megaads informed in xxxxxxxxxxxxxxxxxxxxxxx
6.- Aplication will display in same window form “Your message has been sended” with a button OK,
once user press it, aplication will close form. In Spanish “Tu mensaje ha sido enviado”.

When Antiscrape has been triggered … and in Page “Antiscraping > TMRPS Management”
…Case B is selected , obviously …
in current session user the step 3 will not be do by aplication.
EmailMessage that will receive owner Ad

From: no-reply-email@megaads.com *1 From: no-reply-email@megaads.com

Subject: Title AD Subject: Title AD

Message: Message:

Dear user, we send you a message and the contact details of a Link new Estimado usuario, le enviamos un mensaje y los datos de contacto
registered user who is interested in your advertisment. window / tab de un usuario registrado que está interesado en su anuncio.
to Ad
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Praesent consequat ligula finibus, malesuada sem ac,
Praesent consequat ligula finibus, malesuada sem ac,
volutpat lacus.
volutpat lacus.
Aliquam sed hendrerit ipsum. Nunc porta, libero non
Aliquam sed hendrerit ipsum. Nunc porta, libero non
fermentum elementum, justo quam imperdiet magna, vel
fermentum elementum, justo quam imperdiet magna, vel
finibus erat urna ac tortor. Etiam tincidunt ligula nec odio
finibus erat urna ac tortor. Etiam tincidunt ligula nec odio
mollis, quis auctor arcu rutrum.
mollis, quis auctor arcu rutrum.

You can reply using the contact email address he provided us with : Puede responder utilizando la dirección de correo electrónico de
user@email.com contacto que nos ha facilitado : user@email.com

if you consider that your message is not correct due to its content Si considera que su mensaje no es correcto por su contenido y/o
and/or form, you can let us know so that we can take the forma, puede hacérnoslo saber para que tomemos las medidas
appropriate measures within our reach.: central@megaads.com *2 oportunas a nuestro alcance.: central@megaads.com

Do not reply this email. No responda a este correo electrónico.

*1 & *2 : Emails from megaads detailed in nexts slide requeriments


Email Message that will receive user that he has sended to owner Ad

From: no-reply-email@megaads.com

Subject: Title AD

Message:

Dear user, we are sending you a copy of the


message you have sent to the registered user
regarding his advertisement.

Lorem ipsum dolor sit amet, consectetur


adipiscing elit. Praesent consequat ligula
finibus, malesuada sem ac, volutpat lacus.

Aliquam sed hendrerit ipsum. Nunc porta,


libero non fermentum elementum, justo quam
imperdiet magna, vel finibus erat urna ac
tortor. Etiam tincidunt ligula nec odio mollis,
quis auctor arcu rutrum.

We have provided your email account


user@email.com so that both of you can maintain
communication.

Do not reply this email.


Email Message that will send to MegaAds

From: no-reply-email@megaads.com

Subject: Email Message by User-Id to User-ID about Ad-Id / Country : Country City: City

Message:

Sended by User: User-ID


IP: xxx.xxx.xxx.xxx Timestamp: yyyy/mm/dd hh:mm:ss
email : userinteresed@email.com
Country: Country Region: Region City: City

To user: User-ID
email : userownerad@email.com
Country: Country Region: Region City: City
Owner Ad: Ad-Id Modality : Paid/Free
Type: Services / Products Class: Offer / Demands

Family: FamilyName
Subfamily: SubfamilyName

Copy of the message :

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Praesent


consequat ligula finibus, malesuada sem ac, volutpat lacus.

Aliquam sed hendrerit ipsum. Nunc porta, libero non fermentum


elementum, justo quam imperdiet magna, vel finibus erat urna ac
tortor. Etiam tincidunt ligula nec odio mollis, quis auctor arcu
rutrum.
Requeriment to boudget
NOT TO DO NOW
00.- Have in mind that there is pending a
requeriment of this page shared in the
past. If you don’t know wich one is, let me
know.

00.- Option Telephone Customer Service


will be rename with Phones & Emails
Customer Service

00.- Add at the end if the page section


“Message email Services” with fields:

• Email from no-reply:


• Generic email info:

Both text area to contribute emails account

For the aplication process message email an in order to test, you can use emails informed in code.
Once it will tested we will request improvement option Director > Telephone Customer Service
User data
management
I have requested several times somes updates /
improvements to do in option to manage users
without success.

Hope this time the minimum things that are


requested to be done as they are necessary to be
able to test the application.
Editing Users Phones & Searching data info
Manager > Users Management > Users List To do a search by First Name or User Name is not useful.

00.- Keep the mentioned fields but it is really necessary to add the fields:
• Country
• City
• Region
• Email (as now is)
• Phone Number 00.- The search by Phone Number must be
• User-ID apply not only for Main Real Phone Number,
search must do in all phone numbers of users:
Main, secondaries and Fake Phone Numbers!
Manager > Users Management > Users List > Manage Phone Numbers
The aplication must allow just using minimum 3
numbers to do a search and list phone numbers
that containing the “chain” numbers searched.

00.- Add the field “Fake Phone Number” with


the fake Phone number generated by aplication
when the registered user has contributed
and/or modified Phone Number.
+34 619 128 976

Manager > Users Management > Users List > Manage Phone Numbers > Edit User Phone
00.- Add the field “Fake Phone Number” with
the fake Phone number generated by aplication
when the registered user has contributed
and/or modified Phone Number.
Fake Phone Number
619 128 976 Field “Fake Phone Number” is editable too.
Creating User Edit User

00.- Use same format fields & fields ubication for create user than to update user, please, respect ubication fields proposed. Thanks
00.- Add the fields:
• Fake Phone Number, aplication will generate Fake Phone Number as soon as Phone Field is informed and cursor change to other field, .. Or Create is
pressed and Phone is contributed.
• Switchboard Code (numeric 4 digits required mandatory randomly for users with Paid ads), Check Document requeriments, search Word “Phone Code”
• Status User

00.- The field Tickets in Basket must be editable when the user who is editing or creating is Admin or Director. Not allowed for Manager
00.- The Status User allow Staff to check status user and to change it if user is conflictive ¿Do you have the documentation about blocking & banning users?
Creating User Edit User

00.- Use same format fields & fields ubication for create user than to update user, please, respect ubication fields proposed. Thanks
00.- Add the fields:
• Fake Phone Number, aplication will generate Fake Phone Number as soon as Phone Field is informed and cursor change to other field, .. Or Create is
pressed and Phone is contributed.
• Switchboard Code ID & Switboard Code Pass. Switchboard Code ID = Integer 4 digits Unike key as User-Id , Switboard Code Pass = RND Integer 4 digits
Both fields is required for users with Paid Ads. Check Document requeriments, search Word “Phone Code”
• Status User
• Trusted User
00.- The field Tickets in Basket must be editable when the user who is editing or creating is Admin or Director. Not allowed for Manager
00.- The Status User allow Staff to check status user and to change it if user is conflictive ¿Do you have the documentation about blocking & banning users?
00.- The Trusted User to manage is user is Trusted or not. Remeber, Ads from trusted user can be published without a review from Staff.
About Phone Code ( Switchboard Code ID & Switchboard Code Pass )
Director > Telephone Customer Service Until Switchboard will not be available for registerd
users, value “Show Phone Code” will be “No”.

Once Switchboard will be implemented, staff will


change value and fields:
Switchboard Code Id & Switchboard Code Pass will be
displayed to users when they have Paid Ads current or
non current.

Public Site Page “My Profile” is not


working.

I can not take a printscreen to explain


detail ino about swichboard fields use.
have been understood and
have been clarified all doubts?

If yes, we can continue…


So, now, next step is to know
exactely in wich situation is the
aplication.
Your mision and objective

Now that you have understood the requirements that were made, you are ready to:

A.- Meet with Technology to find out to what degree the requirements have been fulfilled.

B.- Transfer to me the real situation of the application for decision making. In next slides there
is questionnaire to be completed and returned.

C.- Transfer your knowledge to technology to reconduct the project and bring it to the right
direction.
01.- Radar concept.
Manager > Pub Features > Radius Range > Radius Action Users
01.1- Is the application taking into account the radius of action of the users, differentiating the radius of action of a visiting
user with respect to a registered user who has a larger radius of action?

01.1- YES NO

01.2- Is the application displaying ads that are only within the user's range?

01.2- YES NO
02.- Intensity signal concept.

Manager > Pub Features > Radius Range > Radius Range Subfamily Extended
When option “Activated Control Radius Range Extended Ads” is selected…

02.1- Is the application showing the ads that are only within the user's range to show the ads outside the user's range that
belong to a category with an extended range?

02.1- YES NO

Manager > Pub Features > Radius Range > Offer Demand Radius Range

When option “Activated Control Radius Range Ads” is selected…

02.2- Is the application showing the ads outside the user's range that belong to a Offer & Demands values concepts?

02.2- YES NO
03.- Scope users.

03- Is the application managing the 3 concepts (Radius action, Extended Ranges ads, Offers & Demands Ranges Ads) to
create users scope ads?
03.- YES NO
04.- Calculating Distances

04- The application must not use third party resources for distance calculation. Are you using Haversine's formula to
calculate distances?
04.- YES NO

05.- Listing Ads.

05.1- Is the application prioriting list (sort) of ads to be displayed in base requermients as:

• Paid Ad current certificated


• Paid Ad current non certificated
• Free Ad current certificated
• Free Ad current non certificated

• Paid Ad non current certificated


• Paid Ad non current non certificated
• Free Ad non current certificated
• Free Ad non current non certificated

05.1- YES NO

05.2- Is the application using timestamp last event to sort ads on each group above mentionated?

05.2- YES NO
03.- Scope users.

03- Is the application managing the 3 concepts (Radius action, Extended Ranges ads, Offers & Demands Ranges Ads) to
create users scope ads?
03.- YES NO
04.- Calculating Distances

04- The application must not use third party resources for distance calculation. Are you using Haversine's formula to
calculate distances?
04.- YES NO

05.- Listing Ads.

05.1- Is the application prioriting list (sort) of ads to be displayed in base requermients as:

• Paid Ad current certificated


• Paid Ad current non certificated
• Free Ad current certificated
• Free Ad current non certificated

• Paid Ad non current certificated


• Paid Ad non current non certificated
• Free Ad non current certificated
• Free Ad non current non certificated

05.1- YES NO

05.2- Is the application using timestamp last event to sort ads on each group above mentionated?

05.2- YES NO
06.- Antiscrape
06.1- Is the application counting numbers of Page Details requested in user session differentiating inside radius action
(LRPD-Inside) vs oustside radius action (LRPD-Outside).

06.1- YES NO

06.2- Is the application counting in user’s session the Total Maximun Request per Sesssion (TMRPS) in order to trigger
Antiscrape?
06.2- YES NO

Once Antiscrape is trigered…

06.3- Does aplication react and antiscrape changes options selected in & as:

• Radius Action Users


[ it will Select to Activate Control Radius ]

• Families Radius Range Extended


[ If Extended is not selected, antiscrape will not change option selected BUT if Extended Range is selected… antiscrape
will unselect option ]

• Offers & Demands [ If Offers & Demands Range is selected, antiscrape will not change option selected BUT if Offers &
Demands Range is unselected… antiscrape will select option ]

06.3- YES NO
06.4- Does the application only send one email alert per session user when anti-scrape has been triggered by exceeding
the number of detail page requested?

06.4- YES NO
07.- Antiscrape countermeasures

Admin > Config Public Site > Anti Scraping > TMRPS Management

07.1- In Case A … when LRPD-Inside has been superated by user session, is server response increase session current
user delay in minutes informed ?
07.1-- YES NO

07.2- In Case B ... when LRPD-Outside has been superated by user session, is server showing Fake data to current user
session?
07.2-- YES NO

07.3- In Case C … when TMRPS has been superated by user session, is server response increase session current user delay
in minutes informed ?

07.2-- YES NO

07.4- In Case D … when TMRPS has been superated by user session, does aplication activated Radius Control Users ?

07.2-- YES NO
07.5- In Case E … when TMRPS has been superated by user session, does aplication Ban IP & Show fake Page Server Error ?

07.2-- YES NO
08.- Reposition Ads
08- When ads has been respositionated Automatically and/or Manually, how method are they listed ?

08.1- Equitable Agresive

09.- About Fake Data information ¿Has been implemented functionality to display fake phone numbers when antiscrape is
active and TMTPS has been exceeded?

09- Equitable Agresive

02.- About key cod Phone (switchboard) … ¿Has been implemented functionality to generate code and display it for
registered users in his profile?

10- Equitable Agresive


About Ads Search, Priority Ads List,

Page 24 Page 25 Page 27


About Ads Search, Priority Ads List,

Page 28 Page 32
About Radius Action Users, Offers & Demands Ranges, Subfamilies Extended Ranges

Page 57 Page 58 Page 59


About Radius Action Users, Offers & Demands Ranges, Subfamilies Extended Ranges

Page 60 Page 61
About Phone Code, KeyPhone & Switboard in document requeriments

Page 94 Page 99 Page 101


About Fake Data Info

Page 105 Page 106


About Review Ads / CAT18YO

Page 34 Page 35
About Default Fam & Default Search

Page 70 Page 71 Page 72


About Exception Ip

Page 88 Page 89
About AntiScrape Extreme Caution Action

Page 104
About Renewal & Repositions Ads

Page 129 Page 130


About Renewal & Repositions Ads

Page 131 Page 132

You might also like