WEBVTT

NOTE
Subtitles for "Introduction to descriptive modelling of the World Wide Web.webm", a video by Renepick (own work, 23 November 2016),
licensed under the Creative Commons Attribution-Share Alike 3.0 Unported license: https://creativecommons.org/licenses/by-sa/3.0/deed.en
Video page on Wikimedia Commons: https://commons.wikimedia.org/wiki/File:Introduction_to_descriptive_modelling_of_the_World_Wide_Web.webm
ShareAlike (as stated on the Commons page): if you remix, transform, or build upon the material, you must distribute your contributions under the same or compatible license as the original. Changes made: subtitles added.
These subtitles are a transcription made by AI and checked by a second AI agent. No human reviewed them.

00:00:00.390 --> 00:00:03.900
In this unit, we're going to see
our first really, really small

00:00:03.920 --> 00:00:06.450
descriptive model for the World Wide Web.

00:00:07.100 --> 00:00:09.400
The reason why it's being so small is that

00:00:09.420 --> 00:00:12.470
descriptive models are being
tackled in the next lesson

00:00:12.750 --> 00:00:16.160
quite extensively and
we just want to see one

00:00:16.180 --> 00:00:18.670
really small model in order to see

00:00:18.830 --> 00:00:20.710
what we can do with the generative models

00:00:20.920 --> 00:00:22.710
so we keep it really small here.

00:00:24.100 --> 00:00:27.760
When you complete this unit,
you should be familiar with

00:00:27.780 --> 00:00:30.720
two simple descriptive models of the web

00:00:30.740 --> 00:00:32.450
as a collection of text documents

00:00:33.220 --> 00:00:34.920
and you should be able to discriminate

00:00:34.940 --> 00:00:36.790
between the model and the reality.

00:00:37.740 --> 00:00:40.540
You should be familiar with the
concept of a modeling choice.

00:00:40.560 --> 00:00:43.310
We show you one small example already

00:00:43.560 --> 00:00:47.230
and you should be able to therefore
criticize your descriptive model.

00:00:48.000 --> 00:00:50.340
And last but not least, we will give you

00:00:50.360 --> 00:00:53.210
six important steps for
descriptive modeling.

00:00:54.930 --> 00:00:58.020
So let's build a very simplistic
model for the World Wide Web.

00:00:58.040 --> 00:00:59.410
So we take a web crawl,

00:00:59.860 --> 00:01:01.240
in our case only a part of it,

00:01:01.260 --> 00:01:02.890
namely the Simple English Wikipedia,

00:01:03.340 --> 00:01:05.560
but obviously we could
have crawled much more,

00:01:05.580 --> 00:01:07.370
I mean, as long as we have computers.

00:01:07.900 --> 00:01:10.830
And we use the collection
of text documents model.

00:01:11.720 --> 00:01:14.800
And our goal, as before, is
to measure the size, because

00:01:14.820 --> 00:01:16.680
we wanted to know how big
is the World Wide Web,

00:01:16.700 --> 00:01:19.510
or in our case how big is
the Simple English Wikipedia.

00:01:20.740 --> 00:01:24.810
And the idea is we could count
words as a measure of size.

00:01:25.000 --> 00:01:27.180
And we make an assumption here of saying

00:01:27.200 --> 00:01:30.310
well, everything between
two successive white spaces

00:01:30.620 --> 00:01:32.330
is considered to be a word.

00:01:33.320 --> 00:01:35.870
So we have a web crawl and
we want to know the size,

00:01:36.330 --> 00:01:39.360
but from the web crawl we could see it as

00:01:39.380 --> 00:01:40.520
a collection of documents

00:01:40.540 --> 00:01:42.020
and there we could count the words

00:01:42.040 --> 00:01:45.630
to give us an indicator
of what the size is.

00:01:46.300 --> 00:01:49.930
And when we run this through, as
we will also see in the next video,

00:01:50.440 --> 00:01:53.910
the result is that we have
about 1.2 million words,

00:01:54.480 --> 00:01:56.750
at least in my version of the crawl

00:01:57.240 --> 00:02:01.110
and might be different to
what you see in the exercises,

00:02:01.640 --> 00:02:05.880
so let's already criticize our
model and our modeling choices.

00:02:05.900 --> 00:02:09.960
I want you to think about
this now for a second,

00:02:09.980 --> 00:02:11.970
so maybe you can even pause the video

00:02:12.260 --> 00:02:15.440
and think about something
that could have changed

00:02:15.460 --> 00:02:18.890
the result of 1.2 million
words that we have got.

00:02:19.540 --> 00:02:21.970
So let's go out and criticize our model.

00:02:22.080 --> 00:02:24.140
Therefore I would give you two sentences

00:02:24.160 --> 00:02:27.050
and ask you how many words
are in these sentences.

00:02:28.520 --> 00:02:30.840
So the sentences are “I live in Koblenz”,

00:02:30.860 --> 00:02:33.290
where our MOOC is being held

00:02:33.700 --> 00:02:36.610
and the other is “I
live in San Francisco.”

00:02:37.160 --> 00:02:39.540
and as I have already
noted down the answer

00:02:39.560 --> 00:02:41.140
in the first case is four words

00:02:41.160 --> 00:02:43.210
and in the second case is five words.

00:02:43.820 --> 00:02:47.260
And we can see that
the result varies a lot

00:02:47.280 --> 00:02:49.530
on the implicit choice that we made.

00:02:49.960 --> 00:02:52.090
Because some people might argue that

00:02:52.320 --> 00:02:54.830
Koblenz and San Francisco are both cities

00:02:55.160 --> 00:02:57.850
and in both cases should just be one word,

00:02:57.980 --> 00:03:00.310
so that both sentences
have just four words.

00:03:00.960 --> 00:03:04.230
So what was this implicit modeling
choice that we have made here,

00:03:04.560 --> 00:03:08.300
which was by the way the same
modeling choice that we have made

00:03:08.320 --> 00:03:10.770
when we counted the words on
the Simple English Wikipedia.

00:03:11.600 --> 00:03:14.000
And probably you're
right in realizing that

00:03:14.020 --> 00:03:16.880
the modeling choice was
that everything between

00:03:16.900 --> 00:03:20.510
two consecutive white spaces
should be considered a single word,

00:03:20.600 --> 00:03:23.690
so making San and Francisco
two distinct words.

00:03:24.260 --> 00:03:26.720
Another way that we could have modeled

00:03:26.740 --> 00:03:29.030
the World Wide Web or the
Simple English Wikipedia

00:03:29.450 --> 00:03:32.710
is to model documents instead
of words to measure its size.

00:03:33.080 --> 00:03:36.590
So we could have said that every
URI corresponds to a document

00:03:36.900 --> 00:03:38.820
and then we could have
just counted the URIs

00:03:38.840 --> 00:03:41.170
or we could have counted the
files that we have downloaded.

00:03:41.940 --> 00:03:43.950
But what about duplicate content?

00:03:44.220 --> 00:03:48.070
Which means we have two URIs
that have the same HTML file.

00:03:49.080 --> 00:03:50.970
And what about dynamic web content?

00:03:51.400 --> 00:03:55.190
It is easy to have an infinite
amount of URIs on one domain,

00:03:55.680 --> 00:03:59.900
for example a web server or a
website depicting prime numbers

00:03:59.920 --> 00:04:02.950
and having all the prime
numbers as different URLs.

00:04:04.200 --> 00:04:09.360
So you see that already a simplistic model

00:04:09.380 --> 00:04:12.170
where we just count
the amount of documents

00:04:12.660 --> 00:04:17.050
might be pretty hard to follow through.

00:04:17.420 --> 00:04:20.290
In Simple English Wikipedia
I guess the answer would be

00:04:20.700 --> 00:04:23.710
something like a hundred
thousand documents.

00:04:24.140 --> 00:04:26.930
So what is important
to keep in mind is that

00:04:27.440 --> 00:04:31.270
the instantiated model reflects a
particular situation in the world

00:04:31.730 --> 00:04:33.760
when we take a collection of web pages

00:04:33.780 --> 00:04:36.090
in order to build a text model,

00:04:36.470 --> 00:04:40.370
our model characterizes how the
world might work in general,

00:04:40.860 --> 00:04:42.580
but the models we study only have

00:04:42.600 --> 00:04:45.190
a special snapshot of a special situation.

00:04:45.340 --> 00:04:48.710
If we take the crawl at two hours later,

00:04:48.920 --> 00:04:51.160
the results already
might vary a little bit.

00:04:51.180 --> 00:04:52.840
And it's also important to understand

00:04:52.860 --> 00:04:54.920
that the web is a highly
dynamic environment,

00:04:54.940 --> 00:04:59.730
so it's really hard to model this
with a snapshot of the web anyway.

00:05:01.360 --> 00:05:03.400
I want you to keep in
mind six important steps

00:05:03.420 --> 00:05:06.450
for studying objects
with the help of models.

00:05:07.060 --> 00:05:09.560
So the first thing that you should do is

00:05:09.580 --> 00:05:11.540
you should select an object of study.

00:05:11.560 --> 00:05:13.660
In our case you could
select the World Wide Web

00:05:13.680 --> 00:05:15.170
or the Simple English Wikipedia.

00:05:16.280 --> 00:05:19.680
And then you could, if it's too difficult,

00:05:19.700 --> 00:05:22.130
so for example the World
Wide Web might be too big,

00:05:22.500 --> 00:05:24.390
you could select a toy example,

00:05:25.000 --> 00:05:27.930
so maybe a certain web crawl or
the Simple English Wikipedia.

00:05:29.080 --> 00:05:32.700
And then you should select a
perspective to look at this object.

00:05:32.720 --> 00:05:34.840
In our case, when we look at the web,

00:05:34.860 --> 00:05:37.180
you could say, well, let's look at the web

00:05:37.200 --> 00:05:38.930
as a collection of text documents

00:05:39.690 --> 00:05:41.840
and then you select a way of modeling,

00:05:41.860 --> 00:05:44.690
so descriptive or
generative or predictive,

00:05:45.490 --> 00:05:47.060
and then you make your model assumptions,

00:05:47.080 --> 00:05:50.450
so in our case how did we define a word.

00:05:50.700 --> 00:05:53.260
And it's even the best when you explicitly

00:05:53.280 --> 00:05:54.680
write down your model assumptions,

00:05:54.700 --> 00:05:57.010
especially when you later
want to create a paper

00:05:57.220 --> 00:05:59.510
so other people understand
what you really did.

00:05:59.840 --> 00:06:02.060
And it's always important to be aware

00:06:02.080 --> 00:06:05.160
of all the selections and
choices and assumptions

00:06:05.180 --> 00:06:09.080
that you made in order to be
able to critically discuss them

00:06:09.100 --> 00:06:11.050
or to critically discuss your model.

00:06:11.260 --> 00:06:14.230
I mean, this is the really
scientific part that comes into mind.

00:06:14.580 --> 00:06:17.410
We will see more examples
of this during this course

00:06:18.120 --> 00:06:20.740
and I hope this information
was useful for you

00:06:20.760 --> 00:06:22.600
and you will enjoy the rest
of the Web Science MOOC.

00:06:22.620 --> 00:06:23.670
I'm Rene Pickhardt speaking.
