splunk CONCEPTS Yew Index sme Processing Splunk reads éata froma sure, uch 35 alee por. on host (eg "my machine’) cave that source ints souretype (e. "sys09" “aecess_combined "apache evar. then extracts estar, beaks Up the source into individual events fg, fog events les), whicn can be asingledine ‘ormultpleines, and writes each event nto an index on ek for ltr retreva wth earch SearchtimeProcestng: When scorch stat, matching indexed event are retrieved fm dtk felds(eg. cace=t04, szee™cavsd) ae extracted fm the events text andthe events lassed by matching against eventype defirivons (eg, ‘ercor', "ogsn',.). The event retumed froma seareh can then be powerfully transformed using Sphink's search anguage to generate repre that Wve fon dashboard. Aneventis single entry of data, Inthe contest offog file this ean eventin a Web activity og More specifica, an event isa set of values aesocited with atimestamp, Wile ‘many events ae short and only take upline or two, others canbe long suchas 3 ele ext document, config le or whole java stack ace. Splunk uses line breaking rules to determine how it breaks these event up for deplay i he search Renae [sources the name ofthe le stream, or ether input ftom ich a particular event coniginates for example, valogimessages or UDP Sources at classed it Sourcetypes, which can ether be wel known such as acess_combined (HTTP Web Server logs}, or am be created onthe fy by Splunk when itseesa source with data and formatting it hast seen before Events with the sare surcetype can come from efferent sources—events fom the fle varogitessages ane trom 2 y0g input on udp can both have surcetype= tn ys09 irae [hosts the name of he physical ova device where an event originate, Host prowdes an easy way afin al dat oginating fem given eve ives When you ad data to Splunk Splunk processes it breaking the data it incividual ‘events timestamps ther, an then stores thers in an inde that it can beater searched and analyzed. By default, data you fed to Splunk is sored inthe "main inex ut you can create and spect other inderes fr Splunk to use for diferent data inputs fies elds te searchable namevalu pangs in event data As Splunk process events at index time ane search time automaticaly extracts fields. ATindex time Splunk fntracts 2 smal tof cefal felsfor each event, ncuging het sourced Soureype.Atsearch ime, Splunk ntracts what can be 8 wide range of fl from ‘he ever dts, including user-defined patterns as well 35 obvious field namefvalue pais such ar wie ide, mn Tags are alaes to eld values. For example if there are wo host names that ref tothe sme computer you could give bot of thore hort values the same 99 (69. "hallooD0", ane then i you search or that ag fg, "al2000), Spink valet vets involving bath Nast name values Eanes Eventyper are croseferenced searches that categorie event st search time Forexample you have defined an eventye called problem that hs seach 200),afeld having ay value fornovalie(eg..cede-* or SOP coce=") Forexampl, the seat coors sourcetype-*access combined” error | top 10 url Renamesa specified ele wildcards can be used tospecily rutile Flas. wireteve indexes acces combined event fom disk that contain the term "or ror" (ANDs are implied between search es), and then for those evens report the top 10 mest common URI values SIs |Asubsearch san argument toa command that runs its own sarc, returning those results tote parent command asthe argument value. Subsearches are contained Insquate brackets. For example finding al sslog events fom the user that ha the pen Tastogin enor sourcetype-sysiog [5 rch login error arn user] Note that the subsearch etuns ane user vue, because by default the comand just retums one value, but there ate options formate fe Relative Time Modifiers Rory Berges using the curtom-time range inte urerintrface you cn pect in yoursench the time nngetofretieved events withthe Latest and earliest —ia Search modifies. The relative tines ate speced with a sting of characters that “ incite amount fine integer and unt! and, optional 3 nap to" me unt @csnap_time_unit> GHneIn eens Foreample: “essor ear Livst~—idid latest—-2n0h" wilretieve events The key tofat searching ito limi the data that need obe pulled off kt an Containing "error" thal ccurted fom yesterday {snapped tomidnight 19th bslute minimum and then ofiterthat data seal as possible in the search so Tasthour {rapped the hou that proceting is done on the iim date necessary. Time Unit: specie as second(s, minute(s) hour) day), week rmonthincr quate) year) "Ume integer defaults 101 (eg. 8” Partition data into separate indexes youl rarely perform searches ar01s multiple Sthesameas pes of dat, For example, pul web dtsin ane index and reall date a anather. Snapping: indicates the nearest or lates time to which your tie amount rounds + Search as specifeally asyoucan (eg fatal error not ero") down, Snaps rounds down othe Inter time not after the speed time. For trample Hits 115900 and you snap to hous (2h. you wll snap te 1100 not + Limitthesime ranger only what's needed fg, -th nots) 2200. You can snap to" a speci day ofthe week: se 20 for Sunday, >for Monday et «Fier out unneeded less soon as posible inthe search, + Fier out results a soon as possible before aeulations + Forreport generating searches use the Advanced Charting view, and ‘notte Flahimeline view, which calculates imetnes, Pee Cee? * on Flashtimetine, um of Discover eld when not needed Cee eee “Make sure your dsk\/O is the fstst you have aveFilter ests te only inelude those with “alin ther raw text and satue-0 Remove dupleates of resus withthe same hostvalve Keep only search resus wiser" field contains Paderessesn the nor routable class (1000018, ‘luster results together sortby ther “duster_count” values and then return the 20 forges clusters in tase) ‘Group results thazhave the same “host snd "cookie’ occur thin 30 seconds ‘ofeach othe and do nathave a pause ‘greater than 5 seconds betwen each “Group results with he same adres (certip) and where the frst result ‘contains ‘signon and theastresut Sortresults by "ip value fnascending ‘order and then by ur value {in descending ores. Retum the lt 20 results nevetse ore. Retum events with uncommon vales Return the maimum "seny" by "sie where sie" broken down into maximum of 10 equal sized buckets. Return manila) or each value of 0 spltby the value of ba same"host” value and et Countof the remoining results, Return the average foreach hour of any unique fils that ends withthe sting “ay (eg, delay, xe rely ee ‘oleate the average value of COU" teach mute for esen hose ‘create atimechart ofthe count of from “web? sources by "host" Return the 20 most common vals of the ule Return the least common values ofthe “wr fle ans Kay Set velocity to cstance time [exact“tronand to feldrusng | regular expressions. araw event Contains From Susan To: Dav then |tomesusanandtendavd. | Save the running total of"count” ina fel called “total _count compute he dfeence between count fn ts previous vale and tore the [restincounai | Keep the host" and "peas and splay themin the order “host ip Remove the “host anc ip fel Rename the jp field as"Padéess’ ‘Change any host vale that ends with the ecient fed inte a singe value Separate he values ofthe vecoient* Fela into multiple ela values, oplying the top recplents mukvale ela "ecpient” For each esul that dental xcept for that RecordNumiber combine them, setting RecordNumber to bea mul= Find the firstemalladdres inthe recipient fla Find allecpient values thatendin.net ororg Find the combination ofthe valves of foo, "bar and the values of bse Lookup the vale ofeach events ser Teldin the okup table usertogioup, setting the events group fla ‘inte the search results tothe lookup fie users Readin the lou fe "urerscov a2 search results ‘Create new resus for each valve ofthe ‘alia fle withallthe varying values Find the index ofthe frstrecpient valve velocity=distance/ ears = top Peciptenerrr eer ene een! cote eRe wean eren ne mance a er tater een ern et aE ese Pelarct nomic tee eerie Ta) FUNCTION | DESCRIPTION Returns the absolute value of X | Takes pavs of arguments and where arguments ae Boolean || expressions hat when evakated te TRUE return te cerespondingY Celing ofa number | eens adreses that oolong toe potculr subnet Returns the fst valve that «not nll ‘rakater an expression Kung double precson fasting pont Returns the floor ofa umber X 1 evaluates to TRUE the rests the second gument PK evaluates to FALSE the sult evaluates tothe ted argument 2 4£00,¥,2) Returns TRUE Xe Boolean, ens rainoinegee “[rewmsTRUEXenoNUL Ganuld (X) | Rew TUF MXu NUTT Returns TRUE Xa number ST -shistncton setae se chasGariesgholaseg TT This fonction returns the character lenginof a sing X | TReurns TRUE and ony fs tke the Sate pater in I Returns naturaiog, I Retumns the log ofthe frst argument Kwang the second argument 3 thease, Y defaults 010, Lower () | Returns the loverase of iz | Rewarns Xwith the characters in wimmed rom the lcs eto to spaces anata, Log (x,¥) Ltria (x,y) BaLGHGE TT teas muti eer pene max (X,.) | Retwnsthe max ‘m5 (X) | Returns the MOS hash ofa stng value EBCEPEVE| Rewrnsthemin ‘BvcounE (3) | Returns the numberof values afk ‘mv ter (36) | Fitersa multovalued field based on the Boolean exaression Betsy asubset ofthe mulled eld X om stan postion os] Bored V0? peers | | ‘Given a mulrvalued eld and sting delimiter andj the wissen) | | mvjoin (x, ¥) Retuns the current time represented n Unk tine This fonetion tes no arguments and rewune NUL Given two argument Fields X and and vetums the Xf the arguments [returns the constant fe eww Xt Return pseudo-andom number ranging fom to 2147483647 ‘Given epochtime sme Xana relstve tne specer, retune the spochtime vale af¥ applied to ox) replace (X,Y, 2) sting Formed 2 she recur value would Be regex sting in sting 2/1/2009: replace (gate, "*(\a(1,2)) Naty 21/%) "2A a | Fewurs X roundedto the amount of decimal places spected by The Tetons with the choracters in Winmed ram the ight de ztrim(X,¥) | irvisnot specified, spaces and tabs are wimmed, round (X,¥)vaio ‘searchmatch (X) split (x, "¥") stzftine (x,y) strptime (X,Y) substr (X,¥,2) tonumber (X,¥) tostring (X,Y) typeof (x) upper (X) urldecode (x) validate (X,¥,_.) Given) last (0) List 00 Poa out fetuns tue the event matches the search sting X Returns Kas a mltrvalved held spltbe delimiter. Fetus epachtime value Xrendered using the format species by “Given atime represented by a sring X retuns value parsed from | format. Returns a substring field X from start position (i-based) ¥ for {optional enaracters “Converts input sting X toa number, where Y (optional defaults to 10) defines the base ofthe number to convert. Returns a eld vale of asa string I the value isa number reformatsitasa string if Boolean value ether True or Fase Is ‘Sromber he second argument Viz optional ana can ether e “hex” [convert X to nexadecrmal "commas" format X wth commas and 2 decimal places) oF “duration (converts seconds X to readable me format 0h Returns Xwith the characters in¥ trimmed rom both sides isnt pected spaces an tabs are immed, Returns the uppercase of ‘apper username Retuns a sting representation of ts ype fetus the URL X decode “Given pars of arguments, Boolean expressions Xan strings returns int iport), “#RROM: Pore the sting corresponcing to the fist expression X that evaltes neeger", por: >=\1 AND port <= 6: the and deals NUL all are re ERROR: Por oman statistical unctions used with the chart stats, and tmechart commands. Feld names Sere ane nec en Erie Tt Pec Returns the ast seen value ofthe Feld Returns the lit fal values of the feloX za mural entry The order ofthe values ects the order ofinput events Returns the maximum vale ofthe ela the vakies of X are non-numeric the maxis found rom lexicographic ordering Returns the midle-mrt vale ofthe fla X Returns the minimum valve of the felX Ihe valves of X ae non-numeric the min foun from lecographc ordering, Returns the mos fequent ale ofthe Feld X Returns the Xth percentile value ofthe field. For example, er5ttal returns the sth percenle value ofa fla “ota {and min values ofthe field X Returns the sum of the squares ofthe valet of he Fld XREGULAR EXPRESSIONS (REGEXES) ern eer oer eee eer raha \e white space \a\e\a digit pace cig \s not whitespace \as\a alt non whitespace dig \a agit \a\a\a-\a\a-\aia\a\al sn ty nota \\p\D Uteee noni Ww word character (eter number or_)]_Wwiw\w three word chars Ww nota word characte Www three nan-word chars ay ‘sryincluded character Ta-z0-91 ‘any char thatiea thea, Othra9, or Cad no nchuded character (rel any charbutx yore + oneor more \ee integer 2 zevoorone Va\a\a-?\a\a-2\a\a\aa SSN with dashes being options! i or Wie word or digit character (Pevar> | namedenecion (2P\a\a\a-\A\a\-\a\a\a\a) | pullout assWand asignta ssn held ° Tosiealgrouning (AWN) NaI \wore-char then digit OR digit then wordhchar = nartof tne ar Tine begins with at east one digit 3 endofine \aes line ends with stleastone digt ae umber of epetons \a3,5) berween 35 aigts \ escape Mt escape the char ) Tookahead (=\D) error ror" must be preceded by anon-digt negative lookahead (@N\d) error eo Me NERO LON WV ace T Ub) hour leading 225) ‘sveconds itn width Oo ed month name Uae) Worth ram Jaap) Year 008) aisd sb ay 1956-123 January 24,2005 a5 Feb 03 = 2003-0225) ror cannotbe precededby ight splunk> Era EOE Sane lunk.com