Scribd Logo
Upload

Welcome to Scribd!

  • Practical Search in The Cloud

    Uploaded by

    lucidimagination
    37 views18 pages
    Martijn Koster from Lucid Imagination discusses a new cloud-based search service using Solr. The service aims to simplify setup and management of search capabilities. Koster covers the need for the service, how it works, its architecture using Heroku and other cloud technologies, and demos direct access and the API. Lessons learned include automating everything, avoiding lock-in, and using cloud technologies to speed development. The service will be available in an early access program in July.

    Original Description:

    Original Title

    Practical Search in the Cloud

    Copyright

    © Attribution Non-Commercial (BY-NC)

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    Martijn Koster from Lucid Imagination discusses a new cloud-based search service using Solr. The service aims to simplify setup and management of search capabilities. Koster covers the need for the service, how it works, its architecture using Heroku and other cloud technologies, and demos direct access and the API. Lessons learned include automating everything, avoiding lock-in, and using cloud technologies to speed development. The service will be available in an early access program in July.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    37 views18 pages

    Practical Search in The Cloud

    Uploaded by

    lucidimagination
    Martijn Koster from Lucid Imagination discusses a new cloud-based search service using Solr. The service aims to simplify setup and management of search capabilities. Koster covers the need for the service, how it works, its architecture using Heroku and other cloud technologies, and demos direct access and the API. Lessons learned include automating everything, avoiding lock-in, and using cloud technologies to speed development. The service will be available in an early access program in July.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    of 18

    Practical Search in the Cloud

    (sneak preview)
    Martijn Koster, Lucid Imagination martijn.koster@lucidimagination.com

    Wednesday, 25 May 2011

    My Background
    Martijn Koster Sr. Software Engineer at Lucid Imagination
    Search in early Web WebCrawler Customer web services Excite Mail Proxies for Mobile Danger

    Wednesday, 25 May 2011

    What I Will Cover


    A Solr based Cloud Service The need How Demo Wrap up

    Wednesday, 25 May 2011

    Why? Getting Started is Hard


    Many new concepts Complex software configuration System Administration overhead

    Wednesday, 25 May 2011

    Search as a Service
    A few clicks to get started Full Power of LucidWorks and Solr Offload operations Expertise, tools and processes are transferable to your own deployments.

    Wednesday, 25 May 2011

    Usage Model
    Users Web UI LWE JVM Lucidworks LWE JVM User Code Direct Access Solr

    Wednesday, 25 May 2011

    Architecture
    3rd party billing Users Web UI (Rails) Heroku User Code Direct Access (Proxy with URL rewriting) Provisioning Service (Rails) Heroku Provisioner (Ruby)

    LWE JVM LWE JVM LWE JVM

    Control (Chef)

    SQL (RDS)

    Monitoring (Zabbix)
    7

    Wednesday, 25 May 2011

    Deployment Automation
    AMI Chef Custom

    Wednesday, 25 May 2011

    Scaling
    Heroku Direct Access Search Servers Provisioning/DB Monitoring and Control

    Wednesday, 25 May 2011

    Direct Access Example


    The Cloudy Direct Access Solr interface
    Simple submission Simple search First, copy SOLR access URL:

    SOLR="https://access.cloudy.io//solr"

    11

    Wednesday, 25 May 2011

    Example - Add
    $ cat > /tmp/doc.xml <<EOM <add><doc> <field name="id">testdoc_LR2011</field> <field name="text_en">LR rocks!</field> </doc></add> EOM curl $SOLR/my_news/update?commit=true \ -H "Content-Type: text/xml" \ --data-binary @/tmp/doc.xml

    12

    Wednesday, 25 May 2011

    Example - Search
    curl $SOLR'/my_news/lucid? q=news&role=DEFAULT&wt=json' curl $SOLR'/my_news/select? wt=json&q=news&start=0&rows=10'

    13

    Wednesday, 25 May 2011

    Example Lucidworks API


    The Cloudy Direct Access Lucid API interface
    API="http://access.demo.cloudy.io//api

    curl $API/collections
    [ {"name":"lwelogs","instance_dir":"lwelogs {"name":"collection1","instance_dir":"collection1_0"}, {"name":"my_news","instance_dir":"my_news_1"} ]

    14

    Wednesday, 25 May 2011

    Live Demo
    Sam Pierson, Director of Product Development The Web User Interface
    Administration UI improvements Log indexing and graphing

    API access
    Ruby bindings for the LucidWorks REST API http://rubygems.org/gems/lucid_works

    10

    Wednesday, 25 May 2011

    Lessons
    Cloud technologies speed up development Automate everything Avoid lock-in Dogfooding

    16

    Wednesday, 25 May 2011

    Wrap Up
    Search as a Service Built in the Cloud, with open source Full Lucidworks Enterprise and Solr Powered by Lucid Imagination

    17

    Wednesday, 25 May 2011

    When
    Early Access Program: July
    Interested? cloud@lucidimagination.com

    18

    Wednesday, 25 May 2011

    Contact

    Martijn Koster
    martijn.koster@lucidimagination.com http://www.lucidimagination.com/ twitter.com/makuk66

    19

    Wednesday, 25 May 2011

    You might also like