Scribd Logo
Upload

Welcome to Scribd!

  • Topic: Large Scale Index Processing Group Members

    Uploaded by

    Vimalendu Shekhar
    19 views1 page
    This document discusses large scale index processing using distributed transactions and notifications. It introduces Pranav Kulkarni and Vimalendu Shekhar who will discuss updating web indexes as new documents are crawled using MapReduce. However, with exponential internet growth, recomputing the entire index from scratch became impractical. To address this, Google developed Percolator, which is built on Bigtable but adds transactions, locks, and notifications to trigger computation stages. Several papers and articles on related topics are referenced.

    Original Description:

    Paper Propopsal

    Original Title

    Paper Proposal

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    This document discusses large scale index processing using distributed transactions and notifications. It introduces Pranav Kulkarni and Vimalendu Shekhar who will discuss updating web indexes as new documents are crawled using MapReduce. However, with exponential internet growth, recomputing the entire index from scratch became impractical. To address this, Google developed Percolator, which is built on Bigtable but adds transactions, locks, and notifications to trigger computation stages. Several papers and articles on related topics are referenced.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    19 views1 page

    Topic: Large Scale Index Processing Group Members

    Uploaded by

    Vimalendu Shekhar
    This document discusses large scale index processing using distributed transactions and notifications. It introduces Pranav Kulkarni and Vimalendu Shekhar who will discuss updating web indexes as new documents are crawled using MapReduce. However, with exponential internet growth, recomputing the entire index from scratch became impractical. To address this, Google developed Percolator, which is built on Bigtable but adds transactions, locks, and notifications to trigger computation stages. Several papers and articles on related topics are referenced.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    of 1

    Topic:Largescaleindexprocessing

    GroupMembers
    PranavKulkarni
    VimalenduShekhar

    Introduction:
    Updatinganindexofthewebasdocumentsarecrawledrequirescontinuouslytransformingalarge
    repositoryofexistingdocumentsasnewdocumentsarrive.MapReducecanhelptodothisoperation.It
    isaprogrammingtoolthatprocessesproblemsthatcanruninparallel,acrossahugenetworkof
    servers,alsoknownasaclusteroragrid.However,withtheexponentialgrowthoftheInternet,it
    becameimpracticaltorecomputethewholesearchindexfromscratch.Toovercomethis,Googlehas
    comeupwithpercolator.PercolatorisbuiltonexistingtechnologieslikeBigtable,butaddstransactions
    andlocksonrowsandtables,aswellasnotificationsforchangeinthetables.Thesenotificationsare
    thenusedtotriggerthedifferentstagesinacomputation.

    NameofPapers&Articles
    LargescaleIncrementalProcessingUsingDistributedTransactionsandNotifications
    byDanielPengandFrankDabek.
    https://www.usenix.org/legacy/event/osdi10/tech/full_papers/Peng.pdf
    BigDatabeyondMapReduce:Google'sBigDatapapersbyMikioL.Braun
    http://blog.mikiobraun.de/2013/02/bigdatabeyondmapreducegooglespapers.html
    GooglePercolatorglobalsearchjoltsansMapReducecomedownThemachinethatbrews
    theCaffeinebyByCadeMetz.http://www.theregister.co.uk/2010/09/24/google_percolator/
    MapReduce:AmajorstepbackwardsbyByDavidDeWitt
    http://homes.cs.washington.edu/~billhowe/mapreduce_a_major_step_backwards.html
    LargescaleIncrementalProcessingUsingDistributedTransactionsandNotifications
    byDanielPengandFrankDabekandPresentedbyNickRadcliffe
    http://courses.cs.vt.edu/cs5204/fall11butt/lectures/perculator.pdf

    You might also like