The document discusses how to create an Elasticsearch index and specify the number of shards and replicas. It provides an example of using the PUT API to create an index called "test index" with 3 primary shards and 1 replica for each shard. It emphasizes the importance of deciding on shard counts before creating indexes in production to ensure adequate capacity for future data and usage. Index templates are also mentioned as a way to automatically configure settings on new indexes.
Original Description:
Original Title
100 - Adding Indices as a Scaling Strategy English
The document discusses how to create an Elasticsearch index and specify the number of shards and replicas. It provides an example of using the PUT API to create an index called "test index" with 3 primary shards and 1 replica for each shard. It emphasizes the importance of deciding on shard counts before creating indexes in production to ensure adequate capacity for future data and usage. Index templates are also mentioned as a way to automatically configure settings on new indexes.
The document discusses how to create an Elasticsearch index and specify the number of shards and replicas. It provides an example of using the PUT API to create an index called "test index" with 3 primary shards and 1 replica for each shard. It emphasizes the importance of deciding on shard counts before creating indexes in production to ensure adequate capacity for future data and usage. Index templates are also mentioned as a way to automatically configure settings on new indexes.
So once you've decided how many shards your index needs, you need to actually create that index and
00:11.850 --> 00:13.740
specify the shard count that you want.
00:14.460 --> 00:16.620
So let's talk about how to actually do that.
00:19.550 --> 00:24.110
And afterwards, we'll talk about how you can use new indices as a scaling strategy of its own.
00:24.950 --> 00:29.930
So here's an example of how to actually specify the number of primary and replicas that you want on
00:29.930 --> 00:30.710
your new index.
00:31.190 --> 00:34.700
And remember, the number of replicas is applied to each primary shard.
00:35.210 --> 00:37.370
So by saying number of shards ten.
00:37.490 --> 00:43.190
And number of replicas one, that means you're going to end up with ten primary shards and ten replica
00:43.190 --> 00:46.550
shards with one replica for each primary.
00:47.270 --> 00:49.160
So that can be a little bit confusing at first.
00:50.030 --> 00:54.380
Now, in the past, we sort of implicitly created new indices as we've created new data.
00:54.380 --> 00:58.430
And we've just been using the default settings for the number of shards because we didn't care.
00:59.090 --> 01:02.330
You know, we didn't have a lot of traffic in our little course here and we only had one machine.
01:02.330 --> 01:04.850
So it wasn't really worth thinking about too much.
01:05.450 --> 01:08.000
But in the real world, you want to think about this a lot. 01:08.750 --> 01:14.360 So make sure that any new indexes are formally created before you start inserting data into it with
01:14.360 --> 01:17.810
the number of shards that you think you're going to need for the foreseeable future.
01:18.590 --> 01:23.600
Now, to make this easier, there's also something you can look up called index templates, and that's
01:23.600 --> 01:29.630
the way to automatically apply mappings, analyzers, aliases and settings like that to any new index
01:29.630 --> 01:30.470
that gets created.
01:30.740 --> 01:32.150
It can save you some time as well.
01:33.140 --> 01:37.370
So let's actually get some hands on practice in doing this because it is an important thing to remember
01:37.700 --> 01:39.590
and having some sort of muscle memory on.
01:39.620 --> 01:43.970
So let's go off and actually create a new index with a specific number of shards.
01:45.840 --> 01:50.010
So now that we have Cabana, we can use it instead of a console, which is a little bit more friendly.
01:50.130 --> 01:54.270
So let's start by examining the existing settings on the Shakespeare index.
01:54.690 --> 01:58.410
To do that, we can just say get slash Shakespeare.
01:59.200 --> 02:06.180
You see how it even completes for you if you want underscore settings and hit the play button.
02:08.200 --> 02:12.770
So you can see here that by default it only created a single shard with a single replica.
02:12.790 --> 02:15.710
So that's the default on this particular system that we installed here.
02:15.760 --> 02:20.530
So that's how you can find out how many shards and replicas you have on an existing index if you need
02:20.530 --> 02:20.860
to.
02:21.400 --> 02:25.300
Now, let's say we're creating a new index, so we want to explicitly set a higher number of shards
02:25.300 --> 02:25.600
on.
02:25.960 --> 02:30.130
I mean, typically you would want more than one, right, for the ability to actually scale this out
02:30.130 --> 02:30.580
at all.
02:31.240 --> 02:34.990
So let's imagine that we're setting up a new index called Text's Test Index.
02:35.650 --> 02:39.010
So to do that, we would say put slash test index
02:42.280 --> 02:46.180
and then open curly brackets and we would say settings.
02:46.360 --> 02:48.490
Notice how it's doing all the tabs for me automatically.
02:48.490 --> 02:55.660
That's kind of nice number of shards even like lets me autocomplete that it's really handy to use as
02:55.660 --> 02:57.520
compared to just typing it in the hard way.
02:57.520 --> 02:57.730
Huh.
02:58.300 --> 03:03.850
We'll set that to three and we will set a number of replicas also just automatically in there for me
03:04.120 --> 03:04.720
to one.
03:05.650 --> 03:10.000
So that is how you would go about creating a new index with a specific number of shards and replicas.
03:10.030 --> 03:12.550
And remember, you can have multiple shards on one machine.
03:12.560 --> 03:17.740
So by saying number of shards three, that does not require me to have three actual hosts, but it gives
03:17.740 --> 03:20.050
me the flexibility to do that later on if I want to. 03:20.320 --> 03:25.900 So again, you need to think ahead as to how much capacity you think you might need going forward and
03:26.500 --> 03:28.450
try to pick the right number of shards based on that.
03:29.080 --> 03:31.870
Let's go ahead and hit play just to show you that it does, in fact, work.
03:33.430 --> 03:33.700
All right.
03:33.700 --> 03:34.510
That got acknowledged.
03:34.510 --> 03:39.340
So now we can do that same request again to get that information back, make sure that it actually took
03:40.210 --> 03:44.560
test index was what we call that right settings.
03:48.040 --> 03:48.700
And there it is.
03:48.700 --> 03:50.410
Number of charts, three number of replicas.
03:50.410 --> 03:55.090
One, so that is how you go about specifying the number of shards and replicas on a new index.
03:55.420 --> 03:58.510
And in the real world, that is something you will want to do before using it.