WEBVTT

0
00:06.870 --> 00:11.490
Another operational consideration that's very important is setting the heap size
for elastic search

1
00:11.520 --> 00:16.770
because it's almost certainly not set to what it should be.

2
00:16.790 --> 00:20.300
Now the default heap size and elastic search is only one gigabyte.

3
00:20.300 --> 00:24.650
So if you're running on a machine that has 64 gigabytes of RAM, you're only using
one gigabyte of that

4
00:24.650 --> 00:26.220
for elastic search,

5
00:26.300 --> 00:31.040
and if you have things set up by default, the other 63 gigabytes are not completely
going to waste, your

6
00:31.040 --> 00:33.570
OS will use it for disk caching,

7
00:33.740 --> 00:36.480
but obviously this is not the ideal configuration.

8
00:36.650 --> 00:41.810
What you want to do instead is at least give half or less of your physical memory
to elastic search.

9
00:41.810 --> 00:45.910
Generally, you want about half of your physical Ram dedicated to elastic search
itself,

10
00:46.070 --> 00:48.230
and the other have to be given to your operating systems

11
00:48.230 --> 00:52.360
so that lucene and the operating system can just use it for caching purposes.

12
00:52.520 --> 00:57.020
Elastic search is very disk heavy so you want to make sure that has a lot of memory
available to cache
13
00:57.020 --> 00:58.860
what's on your disk into memory,

14
00:58.880 --> 01:02.270
that's why you want to leave about half of your physical memory just to lucene in
the operating

15
01:02.270 --> 01:06.510
system, and only allocate half of it to elastic search.

16
01:06.530 --> 01:11.390
Now if you're not doing any aggregations on analyzed string fields, you know we've
seen examples of doing

17
01:11.390 --> 01:12.980
that earlier in the course,

18
01:12.980 --> 01:16.790
you can get away with even less than half of the memory for elastic search and that
might be a good

19
01:16.790 --> 01:17.890
thing.

20
01:18.050 --> 01:20.590
The more memory you have for caching your data, the better,

21
01:20.600 --> 01:24.170
but obviously, you want to give a classic search what it needs to be efficient.

22
01:24.590 --> 01:28.610
Now smaller heaps are going to result in faster garbage collections as well,

23
01:28.610 --> 01:32.720
so it's not just about getting more memory into your cache, it's also about making
sure garbage collection

24
01:32.720 --> 01:34.450
is running efficiently as well.

25
01:34.790 --> 01:36.950
Elastic search is built on top of Java,

26
01:37.070 --> 01:41.750
and one of the bad things about Java is that it manages memory using garbage
collection, which means

27
01:41.750 --> 01:45.650
every once in a while it has to go through and clean everything up and that can
actually give you a

28
01:45.650 --> 01:48.890
performance hiccup while that's happening.

29
01:48.890 --> 01:53.800
So if you have a smaller heap, that's a little bit faster to do. The way that you
actually set your heap

30
01:53.800 --> 01:55.780
size, is a couple of different ways.

31
01:55.810 --> 02:02.080
One is by exporting an environment variable called ES underscore heap underscore
size, and you can set

32
02:02.080 --> 02:06.160
that explicitly to whatever amount of memory you want to allocate to elastic search
itself,

33
02:06.160 --> 02:10.810
say 10 gigabytes might be a good choice if you're running on a 24 gigabyte system
or something like

34
02:10.810 --> 02:19.240
that, or you can also export yes underscore a Java underscore OPTS, and you can
specify a specific

35
02:19.240 --> 02:24.100
Java fly there for the heap size. It's there if you want to do it add a little bit
of a lower level and

36
02:24.100 --> 02:29.290
then kick off elastic search once that's been set, either way works, just make sure
that you don't cross

37
02:29.290 --> 02:35.440
32 gigabytes for your heap size on elastic search, once you get above 32 gigabytes
then it takes a lot

38
02:35.440 --> 02:41.380
more memory to store pointers to memory because you can no longer fit that in four
bytes. So for efficient

39
02:41.380 --> 02:45.550
performance, you don't want to go higher than thirty two gigabytes because
performance will take a pretty

40
02:45.550 --> 02:50.900
big hit once you cross that line, this is why we said in the previous lecture that
64 gigabytes,

41
02:50.960 --> 02:55.720
it's kind of the sweet spot that allows you to dedicate 32 gigabytes to elastic
search and the other

42
02:55.720 --> 03:01.270
remaining 32 gigabytes to Lucene and the OS, that's the most optimal configuration
to have because you

43
03:01.270 --> 03:07.150
know obviously individual machines are expensive. Ram, again, is your real
bottleneck so you want to use

44
03:07.150 --> 03:11.160
as much RAM as possible and each physical machine. So there's some guidance.

45
03:11.170 --> 03:15.610
Make sure that you do check your heap size and set it to whatever is appropriate
for the hardware you're

46
03:15.610 --> 03:20.070
running on, because the default size of one gigabyte is almost certainly not what
you want.