Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
60 views

Machine Learning Probability Homework

1. This document outlines 9 homework problems related to machine learning and probability theory concepts: - Defining and analyzing properties of probability distribution functions - Calculating probabilities based on conditional information - Properties of variance, covariance, and independent random variables - Choosing an appropriate concept class and objective function for a machine learning problem - Proving properties of σ-algebras and Borel sets - Constructing a probability space and defining random variables for an experiment - Analyzing the limit of a probability as a random variable approaches infinity - Relating the probability of the intersection of events to the limit of individual probabilities 2. The problems cover topics like probability distributions, conditional probabilities, concepts in machine learning like

Uploaded by

Saurabh Kumar
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
60 views

Machine Learning Probability Homework

1. This document outlines 9 homework problems related to machine learning and probability theory concepts: - Defining and analyzing properties of probability distribution functions - Calculating probabilities based on conditional information - Properties of variance, covariance, and independent random variables - Choosing an appropriate concept class and objective function for a machine learning problem - Proving properties of σ-algebras and Borel sets - Constructing a probability space and defining random variables for an experiment - Analyzing the limit of a probability as a random variable approaches infinity - Relating the probability of the intersection of events to the limit of individual probabilities 2. The problems cover topics like probability distributions, conditional probabilities, concepts in machine learning like

Uploaded by

Saurabh Kumar
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

CAP 6610 Machine Learning

Homework 1:

1. (Probability distribution function)


The uniform distribution for a continuous variable x is
1
p(x; a, b) = , a≤x≤b
b−a
Verify that this distribution is normalized (integrats to one), and find
expressions for its mean and variance.
2. (Probability)
Probabilities are sensitive to the form of the question that was used to
generate the answer (Source: Minka.) My neighbor has two children.
Assuming that the gender of a child is like a coin flip, it is most likely, a
priori, that my neighbor has one boy and one girl, with probability l/2.
The other possibilities - two boys or two girls-have probabilities 1/4 and
1/4
(a) Suppose I ask him whether he has any boys, and he says yes. What
is the probability that one child is a girl?
(b) Suppose instead that I happen to see one of his children run by, and
it is a boy. What is the probability that the other child is a girl?
3. (Random variables and variance)
Show that the variance of a sum is var[X + Y ] = var[X] + var[Y ] +
2 cov[X, Y ] where cov[X, Y ] is the covariance between X and Y
4. (Variables and covariance)
Show that if two variables x and y are independent, then their covariance
is zero.
5. (Independent variables and variance)
Suppose that the two variables x and z are statistically independent.
Show that the mean and variance of their sum satisfies
E[x + z] = E[x] + E[z]
var[x + z] = var[x] + var[z]

6. (Concept class/Objective function)


Alex goes to a casino and finds out about a new type of gambling game.

1
In the game, Alex can randomly choose an integer, let’s say x and feed
that number to a machine. The machine in turn runs a program which
takes the given number as input and transforms that number using a fixed
function F : Z → Z (which takes an integer as argument and returns an
integer) and then adds some integer noise  to that and produces the final
output. So given an integer x the overall output of the program is of the
form: y = F(x) + , where x, y,  ∈ Z.
Now, Alex’s job is to predict the output y. Based on how accurate his
prediction is, he gets some reward. It is evident that Alex does not know
the exact functional form of F and hence is planning to employ a ma-
chine learning model to improve his predictions in order to maximize his
reward. Fortunately, Alex somehow has the some information about the
functional form of F that the program uses to generate its output. The
function F has the following property:

F(αx1 ) + αF(x2 ) = F(F(x1 + x2 ))∀x1 , x2 ∈ Z (1)


where α is an integer constant unknown to Alex.
Naturally, Alex chooses the set of all functions that satisfy the equation
1 as the concept class C of his machine learning model. For training the
model Alex plays against the machine N times. Let x1 , x2 , ...xN be the in-
put values in those N trails and y1 , y2 , ...yN be the corresponding outputs
from the machine. The model is trained with those (xi , yi ), i ∈ 1, 2, ..., N
pairs. Alex uses mean squared loss as objective for training the model.
So basically Alex’s model would choose the function G for its predic-
tion, which minimizes the mean squared error over the training data
((xi , yi ), i ∈ 1, 2, ..., N ) among all the functions in C. Formally,

G = arg maxΣN
i=1 (yi − f (xi ))
2
f ∈C

Given a random integer x̂ as input what should be the prediction of the


model after training is done ?
Hint: Plug x1 = 0, x2 = n and x1 = 1, x2 = n − 1. Can you infer the
functional form of F from this?
7. (Probability)
Let X be a set and let A = (An )∞
n=1 whose union is X where all Ai , i ∈ N
are mutually disjoint. You can convince yourself that the set of all finite
or countable unions of the members of A, including the empty set, φ,
is σ-algebra (you don’t need to prove this). A σ-algebra of this type,
generated by taking all the finite or countable unions of a set of atoms

2
(Ai ’s in this case), is called atomic. Prove that the Borel σ-algebra BR is
not atomic.
8. Your friend, Carlsen performs an experiment in which he repeatedly flips
a biased coin until a head appears. He notes down the outcome of each
trial of his experiment. Naturally the set of outcomes (call it Ω) contains
elements like H, TH, TTH and so on (where H denotes the occurrence
of a Head and T is for a Tail). Now Carlsen needs help to equip himself
with the required mathematical framework to deal with probabilities.
(a) Construct a σ-algebra (F) on Ω so that Carlsen will be able to assign
probabilities to (at the very least) all possible singleton sets {ω}
where ω ∈ Ω, and prove that F is indeed a σ-algebra.
(b) Is the set F countable? Justify your answer.
(c) Now Define a probability measure P : F → [0, 1]. Justify why you
think P is a probability measure.
(d) Give a random variable X on the probability space (Ω, F, P ) as de-
fined by you so far. Now use the same probability as defined in (c)
induced by the random variable X to show that

lim P (X ≤ x) = 1
x→+∞

Also calculate the value of E(X) for the above setup.


9. Let A1 , A2 , A3 , ... be a sequence of events. Prove that,
P (limn→+∞ inf An ) ≤ limn→+∞ inf P (An )

You might also like