1
00:00:00,390 --> 00:00:03,900
In this unit, we're going to see
our first really, really small

2
00:00:03,920 --> 00:00:06,450
descriptive model for the World Wide Web.

3
00:00:07,100 --> 00:00:09,400
The reason why it's being so small is that

4
00:00:09,420 --> 00:00:12,470
descriptive models are being
tackled in the next lesson

5
00:00:12,750 --> 00:00:16,160
quite extensively and
we just want to see one

6
00:00:16,180 --> 00:00:18,670
really small model in order to see

7
00:00:18,830 --> 00:00:20,710
what we can do with the generative models

8
00:00:20,920 --> 00:00:22,710
so we keep it really small here.

9
00:00:24,100 --> 00:00:27,760
When you complete this unit,
you should be familiar with

10
00:00:27,780 --> 00:00:30,720
two simple descriptive models of the web

11
00:00:30,740 --> 00:00:32,450
as a collection of text documents

12
00:00:33,220 --> 00:00:34,920
and you should be able to discriminate

13
00:00:34,940 --> 00:00:36,790
between the model and the reality.

14
00:00:37,740 --> 00:00:40,540
You should be familiar with the
concept of a modeling choice.

15
00:00:40,560 --> 00:00:43,310
We show you one small example already

16
00:00:43,560 --> 00:00:47,230
and you should be able to therefore
criticize your descriptive model.

17
00:00:48,000 --> 00:00:50,340
And last but not least, we will give you

18
00:00:50,360 --> 00:00:53,210
six important steps for
descriptive modeling.

19
00:00:54,930 --> 00:00:58,020
So let's build a very simplistic
model for the World Wide Web.

20
00:00:58,040 --> 00:00:59,410
So we take a web crawl,

21
00:00:59,860 --> 00:01:01,240
in our case only a part of it,

22
00:01:01,260 --> 00:01:02,890
namely the Simple English Wikipedia,

23
00:01:03,340 --> 00:01:05,560
but obviously we could
have crawled much more,

24
00:01:05,580 --> 00:01:07,370
I mean, as long as we have computers.

25
00:01:07,900 --> 00:01:10,830
And we use the collection
of text documents model.

26
00:01:11,720 --> 00:01:14,800
And our goal, as before, is
to measure the size, because

27
00:01:14,820 --> 00:01:16,680
we wanted to know how big
is the World Wide Web,

28
00:01:16,700 --> 00:01:19,510
or in our case how big is
the Simple English Wikipedia.

29
00:01:20,740 --> 00:01:24,810
And the idea is we could count
words as a measure of size.

30
00:01:25,000 --> 00:01:27,180
And we make an assumption here of saying

31
00:01:27,200 --> 00:01:30,310
well, everything between
two successive white spaces

32
00:01:30,620 --> 00:01:32,330
is considered to be a word.

33
00:01:33,320 --> 00:01:35,870
So we have a web crawl and
we want to know the size,

34
00:01:36,330 --> 00:01:39,360
but from the web crawl we could see it as

35
00:01:39,380 --> 00:01:40,520
a collection of documents

36
00:01:40,540 --> 00:01:42,020
and there we could count the words

37
00:01:42,040 --> 00:01:45,630
to give us an indicator
of what the size is.

38
00:01:46,300 --> 00:01:49,930
And when we run this through, as
we will also see in the next video,

39
00:01:50,440 --> 00:01:53,910
the result is that we have
about 1.2 million words,

40
00:01:54,480 --> 00:01:56,750
at least in my version of the crawl

41
00:01:57,240 --> 00:02:01,110
and might be different to
what you see in the exercises,

42
00:02:01,640 --> 00:02:05,880
so let's already criticize our
model and our modeling choices.

43
00:02:05,900 --> 00:02:09,960
I want you to think about
this now for a second,

44
00:02:09,980 --> 00:02:11,970
so maybe you can even pause the video

45
00:02:12,260 --> 00:02:15,440
and think about something
that could have changed

46
00:02:15,460 --> 00:02:18,890
the result of 1.2 million
words that we have got.

47
00:02:19,540 --> 00:02:21,970
So let's go out and criticize our model.

48
00:02:22,080 --> 00:02:24,140
Therefore I would give you two sentences

49
00:02:24,160 --> 00:02:27,050
and ask you how many words
are in these sentences.

50
00:02:28,520 --> 00:02:30,840
So the sentences are “I live in Koblenz”,

51
00:02:30,860 --> 00:02:33,290
where our MOOC is being held

52
00:02:33,700 --> 00:02:36,610
and the other is “I
live in San Francisco.”

53
00:02:37,160 --> 00:02:39,540
and as I have already
noted down the answer

54
00:02:39,560 --> 00:02:41,140
in the first case is four words

55
00:02:41,160 --> 00:02:43,210
and in the second case is five words.

56
00:02:43,820 --> 00:02:47,260
And we can see that
the result varies a lot

57
00:02:47,280 --> 00:02:49,530
on the implicit choice that we made.

58
00:02:49,960 --> 00:02:52,090
Because some people might argue that

59
00:02:52,320 --> 00:02:54,830
Koblenz and San Francisco are both cities

60
00:02:55,160 --> 00:02:57,850
and in both cases should just be one word,

61
00:02:57,980 --> 00:03:00,310
so that both sentences
have just four words.

62
00:03:00,960 --> 00:03:04,230
So what was this implicit modeling
choice that we have made here,

63
00:03:04,560 --> 00:03:08,300
which was by the way the same
modeling choice that we have made

64
00:03:08,320 --> 00:03:10,770
when we counted the words on
the Simple English Wikipedia.

65
00:03:11,600 --> 00:03:14,000
And probably you're
right in realizing that

66
00:03:14,020 --> 00:03:16,880
the modeling choice was
that everything between

67
00:03:16,900 --> 00:03:20,510
two consecutive white spaces
should be considered a single word,

68
00:03:20,600 --> 00:03:23,690
so making San and Francisco
two distinct words.

69
00:03:24,260 --> 00:03:26,720
Another way that we could have modeled

70
00:03:26,740 --> 00:03:29,030
the World Wide Web or the
Simple English Wikipedia

71
00:03:29,450 --> 00:03:32,710
is to model documents instead
of words to measure its size.

72
00:03:33,080 --> 00:03:36,590
So we could have said that every
URI corresponds to a document

73
00:03:36,900 --> 00:03:38,820
and then we could have
just counted the URIs

74
00:03:38,840 --> 00:03:41,170
or we could have counted the
files that we have downloaded.

75
00:03:41,940 --> 00:03:43,950
But what about duplicate content?

76
00:03:44,220 --> 00:03:48,070
Which means we have two URIs
that have the same HTML file.

77
00:03:49,080 --> 00:03:50,970
And what about dynamic web content?

78
00:03:51,400 --> 00:03:55,190
It is easy to have an infinite
amount of URIs on one domain,

79
00:03:55,680 --> 00:03:59,900
for example a web server or a
website depicting prime numbers

80
00:03:59,920 --> 00:04:02,950
and having all the prime
numbers as different URLs.

81
00:04:04,200 --> 00:04:09,360
So you see that already a simplistic model

82
00:04:09,380 --> 00:04:12,170
where we just count
the amount of documents

83
00:04:12,660 --> 00:04:17,050
might be pretty hard to follow through.

84
00:04:17,420 --> 00:04:20,290
In Simple English Wikipedia
I guess the answer would be

85
00:04:20,700 --> 00:04:23,710
something like a hundred
thousand documents.

86
00:04:24,140 --> 00:04:26,930
So what is important
to keep in mind is that

87
00:04:27,440 --> 00:04:31,270
the instantiated model reflects a
particular situation in the world

88
00:04:31,730 --> 00:04:33,760
when we take a collection of web pages

89
00:04:33,780 --> 00:04:36,090
in order to build a text model,

90
00:04:36,470 --> 00:04:40,370
our model characterizes how the
world might work in general,

91
00:04:40,860 --> 00:04:42,580
but the models we study only have

92
00:04:42,600 --> 00:04:45,190
a special snapshot of a special situation.

93
00:04:45,340 --> 00:04:48,710
If we take the crawl at two hours later,

94
00:04:48,920 --> 00:04:51,160
the results already
might vary a little bit.

95
00:04:51,180 --> 00:04:52,840
And it's also important to understand

96
00:04:52,860 --> 00:04:54,920
that the web is a highly
dynamic environment,

97
00:04:54,940 --> 00:04:59,730
so it's really hard to model this
with a snapshot of the web anyway.

98
00:05:01,360 --> 00:05:03,400
I want you to keep in
mind six important steps

99
00:05:03,420 --> 00:05:06,450
for studying objects
with the help of models.

100
00:05:07,060 --> 00:05:09,560
So the first thing that you should do is

101
00:05:09,580 --> 00:05:11,540
you should select an object of study.

102
00:05:11,560 --> 00:05:13,660
In our case you could
select the World Wide Web

103
00:05:13,680 --> 00:05:15,170
or the Simple English Wikipedia.

104
00:05:16,280 --> 00:05:19,680
And then you could, if it's too difficult,

105
00:05:19,700 --> 00:05:22,130
so for example the World
Wide Web might be too big,

106
00:05:22,500 --> 00:05:24,390
you could select a toy example,

107
00:05:25,000 --> 00:05:27,930
so maybe a certain web crawl or
the Simple English Wikipedia.

108
00:05:29,080 --> 00:05:32,700
And then you should select a
perspective to look at this object.

109
00:05:32,720 --> 00:05:34,840
In our case, when we look at the web,

110
00:05:34,860 --> 00:05:37,180
you could say, well, let's look at the web

111
00:05:37,200 --> 00:05:38,930
as a collection of text documents

112
00:05:39,690 --> 00:05:41,840
and then you select a way of modeling,

113
00:05:41,860 --> 00:05:44,690
so descriptive or
generative or predictive,

114
00:05:45,490 --> 00:05:47,060
and then you make your model assumptions,

115
00:05:47,080 --> 00:05:50,450
so in our case how did we define a word.

116
00:05:50,700 --> 00:05:53,260
And it's even the best when you explicitly

117
00:05:53,280 --> 00:05:54,680
write down your model assumptions,

118
00:05:54,700 --> 00:05:57,010
especially when you later
want to create a paper

119
00:05:57,220 --> 00:05:59,510
so other people understand
what you really did.

120
00:05:59,840 --> 00:06:02,060
And it's always important to be aware

121
00:06:02,080 --> 00:06:05,160
of all the selections and
choices and assumptions

122
00:06:05,180 --> 00:06:09,080
that you made in order to be
able to critically discuss them

123
00:06:09,100 --> 00:06:11,050
or to critically discuss your model.

124
00:06:11,260 --> 00:06:14,230
I mean, this is the really
scientific part that comes into mind.

125
00:06:14,580 --> 00:06:17,410
We will see more examples
of this during this course

126
00:06:18,120 --> 00:06:20,740
and I hope this information
was useful for you

127
00:06:20,760 --> 00:06:22,600
and you will enjoy the rest
of the Web Science MOOC.

128
00:06:22,620 --> 00:06:23,670
I'm Rene Pickhardt speaking.
