statistical mathematics

Lecture Notes Statistical Mathematics An introduction to difference equations, Special functions, Laplace transform and their applications in statistics Prepared By Mohamed Abdurrahim 1 Contents Chapter: ..................................................................................................................... 4 1.1 Introduction: .................................................................................................... 4 1.2 Difference equations in daily life problems ..................................................... 4 Exercise 1.3: ........................................................................................................... 7 Chapter 2: Difference Calcu ........................................................................................ 9 1.1 The operator ................................................................................................. 9 1.1.1 Properties of the operator ............................................................ 10 1.1.2 Calculating for some elementary functions......................................... 10 1.2 The shift operator ....................................................................................... 16 1.3 The inverse of delta operator ................................................................. 18 1.4 applications: Summation of the series........................................................... 21 Exercise 2.1 .......................................................................................................... 23 Chapter 3: Difference equ ......................................................................................... 26 3.1 Definitions ...................................................................................................... 26 3.2 Linear difference equation............................................................................. 28 3.2.1 Homogeneous linear difference equations with constant coefficient: .. 29 3.2.2 Non homogenous linear difference equation with constant coefficient 34 3.2.3 Method of undetermined coefficients: .................................................. 35 3.2.4 Remark on the method of undetermined coefficients: .......................... 38 3.2.5 First order linear difference equations with variable coefficients.......... 40 3.2.6 Generating function technique ............................................................... 45 3.2.7 Simultaneous linear difference equation ............................................... 48 3.3 Some Non-Linear Difference Equations: ........................................................ 51 Exercise 3.1 .......................................................................................................... 55 Chapter 5: Some Special Functions .......................................................................... 58 5.1 The Gamma Function ..................................................................................... 58 5.2 Application: Family of gamma distributions .................................................. 65 2 Exercise 5.1: ......................................................................................................... 67 5.3 The Beta function ........................................................................................... 68 3 Chapter: 1.1 Introduction: Mathematical computations for daily life problems frequently are based on equations that allow us to compute the value of a function recursively from a given set of values. Such an equation is called “difference equation” or “recurrence equation”. These equations occur in numerous settings and forms, both in mathematics itself and in its applications to statistics, stochastic processes, computing, dynamical systems, economics, biology and other fields. 1.2 Difference equations in daily life problems The following examples have been chosen to illustrate some daily life uses of difference equations. Example 1: if we invest $500 in a bank where interest is compounded monthly at a rate of a year. If we want to know ( ) , namely our wealth at any time ,then we have to solve the difference equation given by ( ) ( ) ( ) ( ) Where at the wealth at the beginning suggest the boundary condition ( ) To deduce the function ( ) explicitly as a function of time we compute recursively for as follows ( ) ( ) ( ) ( ) ( ) ( )) ( continuing in this manner we get ( ) ( ) and from initial condition we have ( ) ( ) By this function we can evaluate our wealth at any given time 4 We observe that the function ( ) satisfies the difference equation. Indeed, if ( ) ( ) then ( ) ( )( )( ) ( ) ( ) ( ) hence is a solution to the difference equation indeed. From last example we can remark that.    the difference equations study the discrete change in the study function(the dependent variable) by a solution to the difference equation we mean to deduce explicitly the variable of the interest as a function of independent variable that satisfy the original difference equation The boundary condition for the difference equations is the initial conditions set on the equation. this conditions help us to determine constants Example 2: it is observed that the decrease in the mass of a radioactive substance from year to year is proportional to the mass that was present at the beginning of the time period. If the half-life of radium is 1600 years, find a formula for its mass as a function of time. Solution: if ( ) represent the mass of the radium after years. Then we suggest the following difference equation ( ) ( ) ( ) or simply, ( ) ( ) ( ) with the boundary condition ( ) ( ) ) ( ) is the discreet change in the mass ( ) , Where, ( represent the constant of proportionality and negative sign indicates the decreasing rate of mass. To find a mass as a function of time we are concerned by means of solving the difference equation we can be done by successive substitution as follows: from if then 5 if ( ) then ( ) ( ) ( ) ( ( ) ( ) ) ( ) Continuing in this manner we have ( ) ( ) And the constant ( ) . thus, ( ) ( ) can be found by using the boundary condition ( ( or, ) ( or, ( ) ( ) ( ) ) ( ) ( ) upon taking the exponential of both sides we get substitute in ( ) then, ) ( ) ( ) ( ) ( )( ) by this function we can deduce the mass of the radioactive material at any time. Another example of mathematical topics in which the difference equations takes place is a Fibonacci sequence. The Fibonacci sequence is the sequence whose general term is the sum of the two previous terms where the first and second terms in the sequence are and respectivly. Namely, is the Fibonacci sequence. If we want to drive the general term of the sequence we are concerned by means of solving the following difference equation ( ) ( ) ( ) ( ) ( ) where, ( ) is the th term of the sequence (general term) The usual way of solving this equation by recursively computing values leads to nothing. So we are in need to more study for different types of difference 6 equations and how to solve them. This is the case we are concerned about in the following chapters. Exercise 1.3: 1- Suppose the population of bacteria in a culture is observed to increase over a fixed time period by an amount proportional to the population at the beginning of that period. If the initial population was and the population after two hours is , find a formula for the population as a function of time. 2- The amount of the radioactive isotope lead pbat the end of each hour is proportional to the amount present at the beginning of the hour. If the half-life of pbis hours, how long does it take for of a certain amount of pbto decay? 3- A body of temperature F is placed at time in a large body of water with a constant temperature of . After minutes the temperature of the body is . Experiments indicate that at the end of each minute the difference in temperature between the body and the water is proportional to the difference at the beginning of the minute. What is the temperature of the body after minute? When will the temperature be ? 4- In 1517, the king of france, francis I, bought leaonardo de vinci’s painting, the “Mona Lisa,” for his bathroom for 492 ounces of gold. If the gold had been invested at an annual rate of (piad in gold), how many ounces of gold would have accumulated by the end of this year? 5- In each of the following, show that ( ) is a solution of the difference equation: ) ( ) ( ) (a) ( (b) ( (c) ( (d) ( ) ) ) ( ) ( ) ( ( ) ) ( ) 7 ( ) ( ) 6- The exponential integral ( ) ( ) is defined by ∫ where is a positive integer. Show that equation , ( ) 8 ( ) ( ) satisfies the difference ( )- Chapter 2: Difference Calculus The calculus of finite differences, in its broad meaning, deals with the change that take place in the value of the function the independent variable, due to changes in the independent random variable. In this chapter the finite difference are studied throw three different operators. Namely, the delta operator , the inverse of delta operator and the shift operator 1.1 The operator Definition 1.1.1: the first order difference of a function ( ) ( is denoted by ) ( ) ( ) and defined by The differences of these first order differences are called the second order ( ) and are denoted by ( ). Thus, differences of the function ( ) ( ( ( ( )) ) ) ( ( ( ( ) ) ( )) ( ( ) ( ) ) ( )) The differences of these second order differences are called the third order differences and so on Note: the symbol is not a quantity but an operator (i.e. a symbol stands for an operation) .thus denotes the repetition of the operator 9 1.1.1 Properties of the operator The following lemma discuss some properties of operator Lemma ( ) The operator ( ) the operator ( ) the operator , and Satisfy the distribution law. i.e. , ( ) ( ) ( ) Satisfy the commutative law with respect to constant , i.e. , ( )( ) Satisfy the index law. i.e. ( ) are positive integers Proof: (i) , ( ) ( ) - ( ) (ii) , , ( ) ( ) , ( ) ( )( ) ( ) ( )- (iii) ( ) ( ) ( ( ( , ( ( ) ) ) , ( , ( ) ( ) ) ( )- ( ) ( ))( ( ) ) ( ) 1.1.2 Calculating for some elementary functions Lemma 1: If and are constants then Proof: ( ( ) ( ) ( 10 ) ) ( ) ) ( ) ( ) - Lemma 2: If and are functions of and the interval of differencing ( ) ( ) ( ) then Or equivalently, ( Proof: ) ( ) ( The first equality can be performed as follows: ( ) ( ( ) ( ) ) ) ( ) The proof of second equality is left as an exercise. We introduce a notation, through the following definition, which is useful in determining the differences of any polynomial function Definition 1.1.2: The product of factors of which the first factor is and the successive factors decrease by a constant difference is called a factorial and is denoted by ( ) , where is a positive integer. Thus ( )( ) , ( ( ) More generally we define, )( ( )( ) ( Examples: with ( ( we have ( ) ( ) ( ) ( ) ( ) ( ) )( ) ) )( )( )( )( )( , ) ( ) - ) An analog to positive factorial notation is the negative factorial notation, which is useful in determining the differences of some rational functions discussed later, introduced by following definition Definition 1.1.3: The reciprocal of the product of factors of which the first factor is and the successive factors increases by a constant difference is called a negative factorial and is denoted by 11 ( ) , where is a positive integer. Thus ( ( )( we have )( more generally we define, )( ( ) ) ( Examples: with ( ) )( ( ( ) )( ( )( ( ) ) ) ) ) )( ( ( )( )( )( )( ( ) ) ) Lemma 3: the first order difference of positive and negative factorial notation is given by ( ) ( ( ) ( ) ) Proof: We have ( ) ( *( ( ( )( ) ) ( * ( ) ( ( ) ) ) ) ( ) , )( ( ) ) ),( ) -+ , ( ) ( ) -+ ( ) )- Moreover, ( ( ) ) ( ( ( )( )( ( )( ( ) ) ) , ( ( ( ( ( ) ) ) ), ( ) - )( ( ) ( ) Last lemma can be easily generalized as given in the following lemma Lemma 3’ ( )( ) ( 12 )( ) ) ( Proof: [is left as an exercise] )( ) )( ( ) Example: Represent the function ( ) and its successive differences in a factorial notation. Solution: We can put the given polynomial into factorial notation form by equating ( ) ( Where, Putting and )( ( ) )( ( ) ) ( ( ) )( ) ( are constants to be determined. we get Similarly, putting and in turn, we find Then Hence, ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Examples: (for ) put the following functions into the appropriate factorial notation then find its first order difference 13 ) 1- ( ) 23) ( ( )( )( )( ( )( ) ) )( 4- ( 56- ( Solution: 1- Clearly, ( ) ( 2- Clearly, ( Now let where and Putting ( ) )( ). Then ( ) ( )( )( ( ) ( ) ( )( ) are constants to be determined. , ( 3- Clearly 4- Clearly )( ( ( ( [( (( )( ( ) )[ ) ( )( ) ) ( ) ) )( )( ( )[ ( ( )( )( ( )( )( ( ) ( ) ) ( ) )( ) ) ( ) ( ) ( ( 5- Clearly )( ) ) then ( ) ( ) )( ( ( then we have ( ) ) )( ( ) thus, 7- Let ) )( )( ) ] )( 14 ) ( ] ) )( ( ] ( ) ) ( ) ) ) ) ) )( ( ) ( )( )( . Then, )( ( ) ( ( ( ) ( ( ) ( ) ) . then ) )( ) then, )( ) ( [ ( )( ) ( ) )] ) Let then we get . Then ( ) ( ) ( ) ( ) ( ) Thus, ( ( )) ( . ( ( )( ( ) ( ) . ( ( ( ( ( ) ( ) )( ( ) ( ) )( ( ) ( ) ( ) ) )/ ( ) ) ( ) ) ( )/ ( ) ( ) ( ) ( ) ) ( ) ) The following lemma is useful to directly determine the th order difference of a polynomial of degree Lemma 4: Let ( ) be a polynomial function of degree defined by ( ) where is a positive integer, are constants and ( ( )) . Then Proof: Since, Then, ( Hence, ( ) ) ( ) ( ) ( ,( ( ) ) ) ( ) - ( ,( ) ) ( ) - Where are constant coefficients independent of . Thus the first ) difference is a polynomial of degree ( To find the second difference of ( ) we have 15 ) ( ) , ( ),( ) ( ) ( ) ( ) ,( - - ) ). The second difference is thus a polynomial of degree( Continuing in this way we find that , ( ( ) )( ) Example: If ( ) Solution: ( )( )( ) and ( ) ( ) is a polynomial in - ( )( ( ) ( of degree ( ) ( ) , find )( ) Hence by the theorem ) This result can also, be obtained by evaluating Note: ( ) ( ) ( ) and then where ( ) is a polynomial of degree 1.2 The shift operator ( ) be a function of the shift operator Definition: Let function is defined as ( ) ( ) and the inverse of shift operator is defined as ( ) ( ) to the Some remarks:   ( ) ( ) More generally we have The shift operator is related to the delta operator as follows: ( ) ( ) ( ) ( ) ( ) ( ) ( ) i.e. 16 ( )   Like Delta operator the shift operator satisfies the distribution law, the commutative law with respect to a constant and the index law i.e. ( ( ) ( ) ) ( ) ( ) ( ( )) ( ) ( ) ( ) such that and are all constants. The shift operator and the delta operator are commutative with respect to each other. ( ) ( ). For i.e. , ( ( ) ( ) ( ) ( ) ) ( )( ) Example Prove that ( the interval of differencing is ) Solution: We have ( ( ) [ ( ( Hence, ( ) ( ( ) ( ) ) ( Hence the formula holds good universally. 17 ( )] ( ) ) ) ( ) ) ) 1.3 The inverse of delta operator Definition: if ( ) and ( ) are two functions of then ( ) ( ) ( ) ( ) Note: satisfies the following properties:  By definition we have  For a constant we have  ( ( ) We calculate now problems Theorem: for then we have ( ) or ( ( )) ( ( )) ) ( ( )) ( ( )) of some elementary functions that’s commonly used in and are constants and the interval of differencing being ) ( )( ( )( ( ) ( ) ( )( ) )( ) Proof: for the first relation we previously showed that ) ) ( )( ( then ] ( ) [( ) hence the answer. The other relations can be proved by same manner. Example: if find the functions whose first order difference are the following functions (i.e. find for the following functions): 123- ( ( )( )( )( ) 4- ) 5- Solution: 18 ) ( ( )( )( ) )( 1- Since 2- Since ) ( . So the required function is ( ) )( ). Then ( )( ) ) (( thus the required function is ( 3- Since previously. Then ( ) ( )( ) So the required function is ( ) 4- Clearly ( ( )( )( )( ( )( ) )( ( )( ) ( ( ) ) ) )( (( )( ) ) ( ) as derived ( ) ( )( ( )( ( ) ( ) )( ) ) ( ( )( ( ) which is the required function 5- Clearly ) ( ( )( ) )( )( ) ( . then ) )( )) . Then, )( ) In the following theorem we introduce a method to evaluate multiplication of two functions. for a Theorem (finding by parts) let and are functions of then for unit interval of differencing ( we have ( ( ) ) Proof: since ( ( ) ( )) ( ) 19 ( ) ( ) ( ) ) Then with ( ) Taking and ( ) ( ) we have, ( ) on both sides we have ( ) By rearranging the terms we have, ( ) Example 1: using find ( ) ( ) for the function ( ( ) Solution: ) ( ) We find by parts as follows, let then we have, ( ) Thus, by parts we have ) ) (( . Where by parts as follows Where ( ( ( ( ) (( . (( ( ) ) ) ( ) ) ( ) ( ) and )/ )/ can be found in the same manner ( ) ( ) ( ) ( ) ( ) ( ) ( ) . ) ( ) ) ( ) ( )/ is obtained by parts one more time as ( ( ) )( ) (( ( ) Then by backward substitution we get ) ) (( ) ) 20 ( ) ( ( ( ) ( ) ) ) ( ) Example 2: (using ) find Solution: clearly ( ) find by parts as follows ( ) ( ( ( )) ( We can see ( ) as follows ( ( )( )( ( ) ( ) Then, ( ) )( )( ) ( ) ) )( ( ) ) ( ) ( ( ( ) ( ) ) ( ( )) can be ( ( ) ( ) ( then we find ( ) ( ) )( )( ( Another method: ( ) for the function )( ( ( ) ) ) )( ) ) ) ) 1.4 applications: Summation of the series As an application of the use of operator we introduce a method to calculate the sum to term of some given series throw the following theorem Theorem 1: let be any function of . Then ∑ where the interval of differencing is , 21 - Proof: ( ). Then Let ( ) Thus, ∑ ( ) , ( ) ( )- , ( ) , ( ) ( )( , ( ) , ( ) ( ( ) , ( )- ( )) Example: use the method of finite differences to find the sum to )terms of the series whose th term is Solution: ∑ ( [ ( ) [ [ ( ( ( Example 2: find ( ) ] ∑ )( ) [ ( )( ( ) ( ) [ ) ( ) ( ) ( )( , ( )( ) ( ) ( ) ] )( ( )( ∑ ( ) )] [ ) ( ) ) ( )( ) ] ) ( ( ( ) 22 ] ( )( ) ] ) Solution: ( ) ) ) - ( ) We have calculated previously that )) ( ( Thus, ∑ ( ) ( [ ( ( ))] , ( ( ) ) ( ) ) (( ( ,( Example 3: prove that ) - )- [ , ( proof: , ( [ [ ( - ∑ ( ) ( ( [ ( ) )( ) ( ), ( ( )] ) ) ) ( ) ( ) ( ) ( ) ( ( - ) )] ) ( ) [ ( ) - )( ( ) )] ( ) ) ( ) )] ( ] )( ) Exercise 2.1 1. Evaluate: () ( ) ,( )( ( ) )( ( ) )( 23 ( ) )- ,( )( ( ) ( ) )( )- 2-prove that if ( ) and ( ) are any functions of () , ( ) ( )( ) , ( )( ) , ( ) ( )( ) * 3. Show that ( ) [ ( ) + ( ) ] ( ) () ( ) 4. if ( ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ) ( ( ) ) ( 5. Show that 1. 2. 3. ( ) 6. if 1. Express 2. Find as a factorial polynomial 7. Show that if ( ) is a polynomial in ( ) then ( ) 8. Obtain a function whose first difference is () ( ) ( ) ( ) ( ) ( ) 9. Express the function and its differences in the factorial notation 24 ) ( ) . Find . 10. Show that if operates on , then and hence that ∑. / . 11. Show that . / / . / . / . / then evaluate ∑ ( ) 12. Use the method of finite differences to evaluate 1. 2. 3. 4. 5. to term (t0 term) 6. (to 13. Sum to terms the series whose th. Term is () ( ( )( )( )( 3. Sum to () ) term) )( ( ) ( )( )( ) terms the series whose ( )( )( ) ) ( ) ( ( ) ( th term is ( ) ( )( )( 4. Evaluate ( ( 7. Show that 1. 2. . .( ( / ) )( ) / )( )( 25 )( ) ) )( ) ) ) 3. . / Chapter 3: Difference equ 3.1 Definitions Definition 1: a difference equation of order is defined as any equation of the form ( ) ( ) ( ) ( )) ( i.e. it is any equation that involves an independent random variable, dependent variable and the successive difference of the dependent variable. ( ) Example 1: order 2 is an example of difference equation of The difference equation may be in shift operator form as ( ) ( ) ( ) ( )) ( or, ( ) ( ) ( ) ( )) ( Example 2: ( ) ( ) ( ) ( ( ) ) ( ) The difference equation may be written also in the subscript notation as follows ( ) Example 3: ( ) ( ) ( ) Definition 2:, the order of the difference equation is defined as the difference between the largest and smallest arguments ,for the function involved, divided by . 26 For example the difference equation given by ( is of order . In fact, the function involved here is ) equation is given by (, Example 4: (for . so the order of the ) ) determine the order of the following function 123Solution: 1- Here, the order is 2 2- The order is 1 3- In this example it’s useful first to put the equation into subscript notation form. since then, Here, the highest argument is and the smallest argument is then for the order of this equation is Definition 3: the degree of the difference equation is defined by the power of the highest successive difference Example 5: in example 4 we notice that the highest successive difference of is and respectively. Since all these differences is with power so the degree of all this equation are . Definition 4: A solution of the difference equation is any function which satisfies the equation. Definition 5: A general solution of a difference equation of order solution which involved arbitrary periodic constants. is a Definition 6: A particular solution is a solution obtained from the general solution by assigning particular periodic constant. Remarks: 27   To determine the arbitrary periodic constants of the difference equation od order we can prescribe independent boundary conditions for the unknown function The problem of determining solutions to the difference equation subject to boundary conditions is called boundary value problem 3.2 Linear difference equation Definition: A linear difference equation of order the form ( ) ( ) ( ) ( ) is a difference equation of i.e. the function and it’s all successive differences raised to power one ( ) ( ) are ( ) Remarks:     If ( ) then equation ( ) is called homogenous linear difference equation (or reduced equation). If ( ) the equation ( ) is called Non-homogenous (or complete equation) Equation * can be written in shift notation form as follows : ) ( ) ( or simply, ( ) ( ) where, ( ) ( ) are all A particular important case arises when ( ) ( ) constants (i.e. independent of ) then is referred as a linear difference equation of order with constant coefficient Example: classify the following equation according to linearity. If linear then classify its homogeneity, coefficient type and the order of the difference equation 12- ( 34- ) = Solution: 28 1- The equation is non-linear 2- The equation is linear non-homogenous with variable coefficient with order 2 3- The equation is linear non-homogenous with constant coefficient with order 2 4- The equation is linear homogenous with constant coefficient with order 3 3.2.1 Homogeneous linear difference equations with constant coefficient: The homogenous linear difference equation with constant coefficient is given by, ) ( ( The general solution for this equation is obtained as follows 1- Let is a solution 2- Substitute in ( ) to obtain corresponding algebraic equation in given by 3- We solve the algebraic equation for to obtain the roots 4- We write a general solution of the difference equation according to the types of the roots as following cases Case 1: roots are all distinct in this case the general solution written in the form Case 2: some of the roots are complex numbers say then we write the solution in the real form given by ( ) where, √ ( ) , are solution, 𝑟 𝜃 𝛼 𝛼 Case 3: some of the roots are equal -if two roots are equal, say , then the solution is written as ( ) -similarly, if the three roots are equal, say , then a solution is ) ( 29 𝑖𝛽 𝛽 -generalizations to the case where more than three roots are equal follow a similar pattern. Example: for the following difference equation (a) find the general solution (b) if find a particular solution for the difference equation (c) find Solution: (a) Let in the difference equation, then it becomes dividing by [assuming ] we obtain the auxiliary equation i.e. or, ( )( ) . Thus two solutions are and . Consequently the general solution is where and are arbitrary constants. (b) by substituting with the boundary conditions solution we get the following equations Solving these equations result in Hence the particular solution is written as (c) From the particular solution directly we have Example 2: for the difference equation given by find (a) the general solution (b) a particular solution if 30 in the general Solution: (a) let Diving on in the difference equation then, we get the auxiliary equation therefore, √ √ Thus, √ and the general solution is written as ( √ ) (√ ) ( (b) Substituting the boundary conditions solution we have (√ ) ( ) (√ ) ( Solving these equations together we get ) √ ( Then the particular solution is given by (√ ) ( Example: (a) Find the general solution of the equation (b) find a solution such that Solution: let dividing by in the difference equation we get then we get 31 ) 𝜃 ) ( √ in the general ) ( )) √ ))( ) From the boundary condition we have ( ) ( ) then the general solution is ( ( Thus, ) ( ) Therefore, the particular solution corresponding to the boundary condition is . Example: / (a) Solve the difference equation (b) find the solution which satisfies the conditions Solution: (a)Let in the difference equation we get Dividing both sides by we get to solve this equation for we note first that satisfies the equation. ). Thus upon using the method of Hence is divisible by ( synthetic long division we get 𝜆 𝜆 𝜆 32 𝜆 𝜆 𝜆 𝜆 𝜆 𝜆 𝜆 𝜆 𝜆 𝜆 𝜆 i.e. ( ( So that the roots are )( )( ( (b) Using the boundary condition have the following equation ) ) )( and the general solution is )( ) ( ) respectively we from which we get . Thus the required solution is ( Example: ) ( ) Find a general solution corresponding to a linear homogenous difference equation if the roots of the corresponding auxiliary equation are given by √ Solution: Corresponding to the repeated roots we have the solution ( ). Corresponding to the repeated roots we have the solution ( ) ( ) corresponding to the single root and the single root we have the ( ) solution ( ) √ corresponding to the complex roots √( ) ( √ so we have the solution we have ( ) Hence the general solution is 33 √ ) (√ ) ) ( ( ) ( ) ( ) ( ) 3.2.2 Non homogenous linear difference equation with constant coefficient Now, that we know how to solve the homogenous difference equation with constant coefficient, we are ready to solve the non-homogeneous equation defined by ) ( ) ( ) ( The general solution of ( 1- ) is obtained as follows: Find the general solution to the corresponding homogeneous ) . we shall equation defined by ( refer to this solution as a complementary solution ( ) 2- Find any solution that satisfies to the complete equation ( ) by the method that will be described later. We shall refer to this solution as a particular solution ( ) 3- Write the general solution of equation ( ) as ( ) ( ) Example: verify that is a particular solution for then write the general solution for the equation. Solution: If then, ( ) ( ( ) ) ( So satisfies the difference equation hence it is a particular solution ( ). To obtain the complementary solution corresponding homogenous equation ( Thus, and ( ) is given by ( ) 34 ( ) we let )( in the to get ) ) ( Therefore, the general solution of is 3.2.3 Method of undetermined coefficients: The method of undetermined coefficients is useful in finding particular solutions of the complete equation ( ) when the right side ( ) consists of terms having certain special forms. Corresponding to each such term which is present in ( ) we consider a trail solution containing a number of unknown constant coefficients which are to be determined by substitution into the difference equation. The trail solution to be used in each case are shown in the following table where the letters are constant coefficient to be determined Terms in ( ) Trail solution Polynomial ( )of degree ( ) or or ( ( ) ) Example 1: Find the general solution of the equation Solution: Putting in the homogenous equation Then and the complementary solution given by ( ) ( ) For the particular solution we assume that substituting in the non- homogenous equation , ( , ) ( ) or, ( ) ( Comparing the coefficient of , we get . By we get ) - respectivly in both sides we gat 35 Or, Hence the general solution is ( Example 2: ) Find the general solution of the equation Solution: For complementary solution we let homogeneous equation given by auxililary equation is given as Then in the corresponding so that the ( )( and the compelemntry solution given by ( ) For a particular solution we assume that homogenous difference equation Dividing both sides by Then ) . Substituting in the non, we get we get and the general solution is given as Example: Find the complete solution of the difference equation Solution: For complementary solution we let in the corresponding homogeneous equation to get the auxiliary equation We observe that is a solution for the auxiliary equation. So by using synthetic long division we get 36 So, ( )( ) ( Dividing both sides by ) ( ) ) in the complete equation. ( ( ) ) , with some algebraic simplifications, we get Comparing the coefficients of respectivly we get and the particulare solution is So the general solution is example: ) and the complementary solution is given by ( ) ( ) To find the particular solution let Then we have ( ( ) ) ( ( ( ) Then )( ( ) ( ) ( ) ( ) Solve Solution: For complementary solution, let equation. So we get Then in corresponding homogeneous and the compelemntry solution is ( ) For particular solution we have to guess a trail solution according the term on the right hand side of the difference equation. Corresponding to the polynomial we may assume as trail solution . Corresponding to the term we may assume the trail solution . So let we get 37 ( ) Or, ( ( , , ) ( ) ( ) - - ) Equating the coefficient of like terms we have Thus and the particular solution is Adding this to the complementary solution, we get the general solution as 3.2.4 Remark on the method of undetermined coefficients: The only requirement which must be met to guarantee success of the method of undetermined coefficients is that no term of the trail solution can appear in the complementary solution. If any term of the trail solution does happen to be in the complementary solution then the entire trail solution corresponding to this term must be multiplied by a positive integer power of which is just large enough so that no term of the new trail will appear in the complementary solution. Example: Solve the equation Solution: For the complementary solution, let homogeneous equation. So we have Then in the corresponding ( and the cmpelmenrty solution is ( ) ( ) ) For particular solution, corresponding to the term we can assume the trail solution since this is not in the complementary solution. 38 Corresponding to the term on the right side of the given difference equation we would normally assume the trail solution this term however is in the complementary solution. So we multiply by to obtain the trail solution . But this also is in the complementary solution. Then we multiply by again to obtain the trail solution . Since this is not in the complementary solution it is an appropriate trail solution to use The particular solution is thus given by Substituting this into the given difference equation we find ( , ) ( - ) , - Or, Comparing coefficient of similar terms we get Then the general solution is given by Example: Solve Solution: ( ) ( ) Let in the corresponding homogenous equation we get Since is a root then by using the synthetic long division we get ( )( ) ( )( ) Then and the compelemntry solution is )( ) ( ) ( ) in the right side For particular solution, corresponding to the term ( of the difference equation we may assume a trail solution . Since 39 these terms occur in the complementary solution we multiply by a power of which is just sufficient to ensure that none of these terms will appear. This is ( ) Substituting this into the so that the trail solution is given difference equation we find ,( ( ) ( ( ) ( ( ) ) ),( ) ( ( ) ( )) ( ) By more simplifications we have So . Thus the required solution is 3.2.5 First order linear difference equations with variable coefficients In this section we are seeking different methods to solve a difference equation of the form ( ) ( ) ( ) ( ) ( ), ( ) where are functions of . The following theorem introduces a general method to solve a first order linear difference equation with variable coefficients. Theorem: The solution of the first order linear difference equation with variable ( ) ( ) , is coefficients, given by where, ∏ ( ) ( where is arbitrary constant. in particular if ( ) given by ∏ ( ) Proof: 40 ( ) ) then the solution is The complementary or homogenous equation corresponding to the given equation is ( ) From this we have ( So that the solution of Taking ( ( ) ) ( ( ) ) is ) ( ) ( ) as the arbitrary constant we have the solution ∏ ( ) We now replace by a function of denoted by ( ) i.e. ( )∏ ( ). And we seek to determine ( ) so that the given equation is satisfied. So by substitution in the complete difference equation we get Dividing by ∏ ( Or, ( ) [ ( ) ∏ ( )] )∏ ( ) ( ) on both sides we get ( ) ( ) ( ) Hence the solution is given by (∏ ( )) * Example: ( ) ( ( ) ∏ ( ) ( ) ) ∏ ( ) ( ( ) ) ∏ ( ) Solve the equation 41 + ( ) Solution: Here, ( ) . then The general solution is given by ∏( ) ( ) ) ( Example: ( ) Solve the equation Solution: Here, ( ) . Then the general solution is given by ( ∏( ) Example: Solve the equation Solution: The general solution t given difference equation is Where, ∏ ( ( ( ) ( ) Thus ) 0 42 . / ) 1 ( ) ) *( ) + Note that an alternative solution to last problem may be attained by considering the last equation as non-homogenous linear equation with constant coefficient [this is left as an exercise]. Example: Solve the difference equation Solution: Multiply both sides of the difference equation ( ) ( ) Hence the general solution is Where, ∏( ( ( ( ) ( ) ) ) ) So, Example: ( Solve the equation solution: The general solution is given by 43 ) where, ) ∏( ( and, ( ( ( Then, the general solution is ( Example: ( ( ) )) ( ) ( ) ) ( ( ) ) ( ) ) ) )( ) Solve the difference equation solution: The general solution is given by Where, ∏( And ( ( ) ( ) ( ( )) ( ) ( ( ) ) ) can be derived by parts as follows 44 ) ( ) ) ( ( So the general solution is ( ( 3.2.6 Generating function technique ) ( ) ( ( ) ) ( ) ) ) Second or higher order linear equations with variable coefficients cannot always be solved exactly. In such cases special methods are used. Among the most common methods is the method of generating function. This method is used to solve linear difference equations with constant coefficient as well. The method is summarized as follows 1- Multiply both sides by and then sum over all values of 2- Define the generating function as ( ) ∑ . by this transform the given difference equation turns into an algebraic equation in ( ) or simple differential equation in ( ) 3- Solve the resulting equation for ( ) 4- Expand ( ) in powers of .as a result, the required solution for difference equation is the coefficient of We will illustrate the use of the method through following examples Example: Use method of generating function to solve the difference equation Solution: On multiplying both sides of the difference equation by all values of we get ∑ Multiply the first sum by . / ∑ ∑ and summing over ∑ and the second by . / . Thus ∑ ∑ 45 Then in terms of generating function , defined by ( ) , ( ) - Solving this equation for ( ) we have ( ( ) ( ) ( ) ( ( ) ) ( ( ) ( ) ( ) ( From the boundary condition we get ( ) ( ) ) ( ) ∑ , we get ( ) ) ( ( )( )( ) ) ) In order to expand ( ) in powers of we simplify ( )first by means of partial fraction. So let Then we get ( Or, )( ( ) ( ) ( ) ) ( ( ) ) Comparing the coefficient of similar terms we get Then thus geometric series expansion we get ( ) Hence the coefficient of ∑( ) ∑ is ∑( ) . So the required solution is Example: Use the method of generating function to solve the difference equation ( ) ( ) 46 Solution: Multiplying both sides of the equation by Noting that ( ∑( ) ∑ ∑( ) ∑ ) and sum over all we get ∑ ∑ we have (∑ ) (∑ ∑ in terms of generating function ( ) we have ( ( ( ) ( ) ) ( ) ) ( ) ) ( ( )) ( ) ( ) ( ) By separation of variable and integrating we get ( ) ( ) ( ) ( ) ∫ ∫ ( ) where is constant. Then ( ) ( ) Then, Where ( ) ( ) ( ) ( is constant. Then expanding ( ) in powers of ( ) ∑. / ) then The general solution is the th term in the ( )’s expansion. Namely, . / 47 3.2.7 Simultaneous linear difference equation If two or more linear difference equations are given with the same number of the unknown functions we can solve such equations simultaneously by using a procedure which eliminates all but one of the unknowns. We illustrate the elimination procedure by the following examples Example: Solve the system Solution: The diven equations can be written in shift operator form as ( ) ( ) Now by operating on ( ) with , then adding the two equations we get ( ) Or, Or, This is homogenous linear difference equation in with constant coefficient which can be solved by letting to obtain the corresponding auxiliary equation given by Then ( )( so that the solution in Substituting in equation ( ) we get ( ( Then ) is given by ( ) ) ) 48 ( Example 2: ) ( ) Solve the following system Subject to the conditions Solution: Putting the given equations in shift operating form we get, Or, ( ) ( ) ( ) ( ) ( ) ), and multiplying equation ( Operating on equation ( ) by ( adding the two equations we get )( ) ( ) (( ) ) then Or, Or, ( ) This is a linear difference equation with constant coefficient and its solution is the sum of the complementary solution ( ) and the particular solution ( ) where for complementary part we assume in the corresponding homogenous equation to obtain the auxiliary equation Then and For particular solution, let we get ( )( ( ) ( ) ) 49 in the complete equation ( ) ( Or, ( ( ) ( ( ) ) ) ) Or, ( ) Comparing the coefficients of similar terms we gat Then, and the general solution is Substituting in equation ( ) we get ( Or, ( ( ) ) ) Where can be found from the initial conditions. Setting respectively we get Or, ( ) ( ) Solving these equations simultaneously we get Thus the required solution is 50 3.3 Some Non-Linear Difference Equations: An important class of non-linear difference equations can be solved by applying suitable transformations which change them into linear difference equations. We illustrate the key idea by the following examples Example 1: Solve the difference equation condition subject to the Solution: Dividing both sides of the given equation by Let now we get then or, ( ) Then ( ) ( To find we use the boundary condition. Since the corresponding boundary condition for or Then So, ) then we have as then, ( )( ) therefore ( ) ( 51 ) Example 2: Solve the difference equation conditions given by subject to the boundary Solution: Taking logarithm of both sides Let the equation becomes This is linear homogeneous constant coefficient can be solved by letting Then, the auxiliary equation is Then ( )( and the solution is Where the constants ) ( ) can be obtained from boundary conditions. Since then the boundary conditions in terms of are Substituting in ( ) we get Solving the equations simultaneously we get Hence, ( ( 52 ) ) Taking exponential of both sides then Example 3: Solve the equation Solution: Taking the logarithm of both sides we get Let then the equation becomes Or, This is non-homogenous linear difference equation with constant coefficient. Its solution is then the sum of two parts the complementary parts and the particular parts For complementary part let in the corresponding homogeneous equation we get the auxiliary equation Then and the solution is For the particular solution, let constant ( ) ( ) in the complete equation. Where is Then and the general solution is ( ) ( ) And the constant from boundary condition. Since terms of the boundary condition 53 then in Substituting in ( ) we get Then ( ) then ( )( ) ( )( Taking exponential of both sides we get (√ ) Example 4: ( ) ) (√ ) . / Solve Solution: Upon taking the logarithms of both sides we get Let then we have The auxiliary corresponding equation is then Then √ Then the solution for is given by ( ( ) √ ( ) ( ) taking the exponential of both sides then ( ( √ 54 √ ) √ ( ) √ ) ) . / Exercise 3.1 1- Write each of the following in subscript notation. ( ) ( (b) ( ( )( ) ( ) [let also evry by ( ) ( ) ) ( ) ] ) ( ( ) ) ( ) ( ) ( ) ( ) and so one . we replace 2- Find the solution of each of the following 1. 2. 3. 4. 5. 6. 7. 8. 3- Solve the following boundary value problems 1. 2. 3. 4. 4- If the roots of the auxiliary equation for a linear homogeneous difference equation are given by 1. Find the general solution corresponding to a linear homogenous difference equation 2. Write the difference equation 5- Find the general solution for the following difference equation 1. 2. 3. 4. 55 for all values of the constants 1. 2. ( and ) 6- Solve the following boundary value problems 1. 2. 7- Solve the following difference equations by generating function method 1. 2. 3. 4. 5. 8-solve the following difference equation subject to given conditions (if any) 1. 2. 3. 4. 5. 6. ( ) 9-solve the following system of difference equations 1. 2. 3. , 10-by means of suitable transformation, obtain the a solution of the following non-linear difference equations 1. 2. 3. 11*- Solve the difference equation 56 ( ) ( ( ) 57 ) Chapter 5: Some Special Functions 5.1 The Gamma Function Definition 5.1(Euler’s expression of gamma function as an infinite integral): the gamma function is denoted by ( ) and is defined by the proper integral ( ) ( ∫ The gamma function has several properties we mention some of them in the following lemma Lemma 5.1: expression of gamma function ( The properties 1234Proof: ( ) ( ( ( ) ) ) √ ) satisfies the following ( ) If is a positive integer Proof of Property 1: from the definition we have ( ) ∫ ∫ Proof of Property 2: from definition we have Integrating by parts we get ( ) ∫ 58 , - , - ) Then ( Or, - , ) ( ∫ ) ( ) Proof of property 3: from property , we have ( ) ( ) ( ) ( ( )( ) ) ( ) Proof of property 4: from definition we have Let ( ) then Setting . / ( ) ∫ ∫ ) ( ∫ ∫ [ ( )] with ∫ we have ( ) ( ) ( ∫ ( ∫ ∫ Transforming the variables )( ∫ ) into the polar coordinate Since the double integral runs over values of ( changes from up to , changes from [ ( )] Let then ) ∫ ∫ ∫ *∫ 59 we get ) in the first quadrant, then up to and we have + [ ( )] ∫ * ∫ , ∫ Or equivalently, + ( ) , - 0 1 √ Remark: the integral ( ) is defined for but ( ) can be extended to the negative values of by a property called analytic continuation. The key ) ( ) allows the meaning to be extended idea is that the relation ( to the interval and from there to , and so on. We illustrate the idea in the following example Example 1: Find ( ) . / Solution: . / . / and . ( ) ( ) Since ( ( ( ) ) ) ( ) . / ( . ( ) or ( ) √ / ) √ ( ) . ( ( ) ( √ /. ) ) / / ( ) ( ) √ we have . /. . / /. /. / √ The gamma function definition ( ) can be used for evaluation some improper integral by choosing the appropriate transformation as discussed by the following examples 60 Example 2: evaluate the integral ∫ Solution: Let then . Thus ∫ ∫ . / ∫ Example 3: evaluate the integral ∫ Solution: Let then ∫ Example 3: √ or ∫ √ , ∫ √ Evaluate the following integral ∫ Solution: Let Since . / then or Therefore, ∫ ( ) ∫ . / 61 therefore ( ) √ ∫ . / . / ∫ Example4: ( ( ) ) Show that for ∫ Solution: Let √ √ we get ∫ ∫ √ √ . / ∫ Example 5: evaluate the following integrals ( )∫ ( )∫ √ ( )∫ ( )∫ √ ( )∫ ( ) then ∫ ∫ ( ) Directly from the definition ( ) Let √ ( )∫ Solution: ( ) Let . / then √ ∫ ( ) ∫ 62 ( ) √ ∫ ( ∫ ( ) Let Let ∫ ( ( ) √ then √ ∫ ( ) Since ∫ ( ) ∫ ( then ( Thus, and ∫ ( ∫ ) ( ) ∫ then ( ) √ √ ( ∫ ∫ ( ) √ )( ) √ ( ) √ ) ( ∫ √ ( ) Let ( ) ) )( ∫ ) ) ∫ ( then ∫ ) ) ( ) Example 6: Find the constant PDF in each of the following functions such that it defines a 63 ( ) ( ) . ( ) ( ) ( ) ( ) Solution: / (1) in order that ( ) represent a PDF it must integrate to one that’s ∫ is an even function then Since the integrand function ∫ then Let Then √ . / ∫ ( ∫ ) with √ √ . Or √ √ So, . Therefore we get ∫ √ ( ) √ and this is the PDF of the standard normal distribution. (2) By same manner we can get mean and variance . Here ( ) is a normal PDF with √ (the verification is left as an exercise) (3) Since Let then ∫ with ∫ . / 64 . / ∫ , ( )- Or, ( ) 5.2 Application: Family of gamma distributions A direct application of gamma function is the gamma family of distributions. Due to the mathematical properties of a gamma function the gamma family of distribution is a sufficient flexible family to play an important role in different areas of statistical modeling. To begin with recall the gamma function as ( ) Then ∫ ∫ ( ) That’s to say the function defined by ( ) PDF. Consider now the transformation get the PDF of the random variable ( ) ( ) defines a for some parameter . Then we as ( ) The resulting PDF is called family gamma distribution Definition 5.2: A gamma distribution with shape parameter and scale parameter ( ) and its PDF is given by denoted by ( ) is ( ) the gamma distribution contains other subfamilies of distributions by special choices of the parameters . For example 65   When appositive integer is then the resulting distributions are called Erlang distribution (important in queuing theory) When the resulting are the exponential distributions  When the resulting is the chi-square distribution with degree of freedom. Lemma 5.2: Let be a gamma distributed random variable with parameters( the th moment of is ( ) ( ) ( ) In particular the mean and the variance are given respectively by ( ) Proof: ( ∫ ) ( ) ( ) [ ∫ ( ) ( ) And ( ) ] ( ) ∫ ( ) ( ( ) Then, Thus, ∫ ( ) ( ( ) ( ) ) ) ( ) ( 66 ( ( ) ) ( ) ( ) ) ( ) ( ) ( ) ). Then ( ) ( , ( )- ) ( ) Exercise 5.1: 1-Prove that ( ∫ ) ( ( ) 2- Evaluate the following integrals: ( )∫ √ ( )∫ ( ) ( ( ( )∫ ) ( )∫ ( ) ) ) ( )∫ ( )∫ ( ) ( )∫ ( ) 4- Prove that ) ∫ ( √ ∫ √ 5-Evaluate the following integrals ( )∫ ( ( )∫ ( ) ) ( ) 67 ( ) ( )∫ √ ( ) 3- Prove that ( ) ( )∫ √ ( ) 5.3 The Beta function Definition5.3: The beta function, denoted by ( ( which is convergent for ) ), is defined by ( ∫ ( ) The following lemma introduces two equivalent useful forms of beta function Lemma5.3: ( ) ( ) ( ) ( Proof: ) ∫ ∫ ( ) ( )Using the transformation, . Thus with ( ( ) By taking ) ∫ ∫ ( ∫ then Moreover, Thus, in equation( . Therefore, ( ) ) ( ( ) ) ( with ( ) . Consequently, 68 ), we get ) ) ( ) ( ∫ ∫ ) ( ) ( ) ( ∫ ) ( ) A fundamental relationship between Gamma and beta function is derived by the following theorem Theorem 5.1: The beta function is connected with the gamma function according to the relation ( ) ( ) ( ) ( ) Proof: Since Let then ( ) ( ) Let ∫ ∫ ( ( ) with . Thus ) ∫ by same manner we have ( ) Therefore, ( ) ( ) ∫ ∫ ( ∫ ∫ Then by using the polar transformation ( ) ( ) ∫ ∫ ( ) ∫ ( ) ∫ we get ∫ ( 69 ) ) ) ∫ ( )( ( Let then √ ) then ( ) ( ) ) ∫ ( )( ( ( ) ∫ ( )( ( ) ∫ ( )( ( Or, )( ( ( Example8: ) ) )) ) √ √ ( ) ( ) ( ) ) Evaluate the following integrals ( ( )∫ ) ( )∫ √ ( )∫ Solution: ( ) ( ) Let ( )∫ ( )∫ √ ( )∫ ( )∫ √ ∫ ( ) then ∫ ( ( ) ( ) ( ) ) . Hence √ ∫ √ ∫ 70 ( ) ( )( ) ( . / . / ) . / (3)by the virtue of the alternative form of beta integral we have where ∫ Then therfore ( ) ∫ ∫ ∫ √ Where ∫ ( )∫ ( ) ) ( ) √ , √ . So . /. /. /. / . / . / ( ) )( ) ( ) ( ) , . Thus . / . / ∫ ) ( ( ( then ( . Then ∫ ( ) let ) ∫ √ Where ( then ∫ √ √ ∫ 71 ( ) ) ( √ Example 9: ) √ Evaluate ∫ Solution: ∫ ∫ Where . Thus ( ( ∫ ) and we have ) ( ) . / . / 5.4 The beta distribution A direct application for beta function is the beta distribution given by the following definition Definition 5.3: A random variable and if lemma 5.4: The function in ( Proof: is said to be have beta distribution with parameters ( ) ( ) ( ) ) defines probability density function By definition of beta function we have 72 ( ∫ ( ) ∫ Lemma 5.5: ( ( ) The th moment of a random variable ( ( ) ( ( ( ) ) ) with ( ) given in ( ) ( ) ) ( ) ) is In particular, the mean and the variance are given by ( ) Proof: ( ) ( ) ∫ ( ( ( ) Therefore, ( ) ( ( ( In particular, Moreover ( ) ) ( ) ( ( ( ) ( ( ( ( ( ) )( ( ∫ ) ) ( ) ) ) ( ) ) ( ) ( ( ) ) ( ) ( , ( )) . )( ) )( ) ( ( )( ( )( ) 73 )( ( ) ( ) ( ) ( ( ) ) ) ( ) ( ) ) )( / ) ) ) ( ) ) Exercises 5.2 1- Evaluate the following integrals ( )∫ ( )∫ ( ( )∫ ( ( )∫ ) ) ( )∫ ( )∫ ( ( )∫ √ ( )∫ ) ( )∫ √ ( )∫ ( ( ( )∫ ) ( ( )∫ √ ) ) ( √ ) ( )∫ 2- Prove that 1. 2. 3. ( ( ( ) ) ( ) . ) ( ( ) ( ) ) / 3-if is a continuous random variable on the rang with PDF ( ) and CDF ( ) show that the following function defines a PDF ( ) ( ) ( ) [Hint: use the transformation ( ) ∫ integrates to 1] ( ), ( )- , ( )- ( ) trying to prove that the integral 74 Chapter 3 : Laplace transforms Definition: Existence of Laplace transforms: Laplace transform for some elementary functions: Example: prove that ( ) ( ( ) ( (2) ( ) ( ∫ ) ) ∫ ( ) ∫ ) ∫ ( ) ∫ ( (3) ) ( Proof: ) ) ( (1) ( [ ( ( ( ( ∫ ( ( ∫ ( ) ) ) ( ) ) ∫ 75 ( ] [ ( ( ) ( ( ) ( ) ( ) ) ) ) ) ) ( ) ] ) ) ( [ ( * ( (4) ( ∫ ∫ ( ) ) ( [ ] )( ( ( ) ) ) ∫ ( ) ( ( ] 76 ) ) + ( ) ) [ ( ) ] ) )