Theory of stochastic objects probability

stochastic processes and inference 1st

Edition Micheas
Visit to download the full and correct content document:
https://textbookfull.com/product/theory-of-stochastic-objects-probability-stochastic-pro
cesses-and-inference-1st-edition-micheas/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

Theory of stochastic objects: probability, stochastic

processes, and inference First Edition Micheas

https://textbookfull.com/product/theory-of-stochastic-objects-
probability-stochastic-processes-and-inference-first-edition-
micheas/

Applied Probability and Stochastic Processes Second

Edition Beichelt

https://textbookfull.com/product/applied-probability-and-
stochastic-processes-second-edition-beichelt/

Applied Probability and Stochastic Processes Second

Edition Beichelt

https://textbookfull.com/product/applied-probability-and-
stochastic-processes-second-edition-beichelt-2/

Fundamentals of probability: with stochastic processes

Fourth Edition Ghahramani

https://textbookfull.com/product/fundamentals-of-probability-
with-stochastic-processes-fourth-edition-ghahramani/
Generalized Statistical Thermodynamics Thermodynamics
of Probability Distributions and Stochastic Processes
Themis Matsoukas

https://textbookfull.com/product/generalized-statistical-
thermodynamics-thermodynamics-of-probability-distributions-and-
stochastic-processes-themis-matsoukas/

Applied Probability From Random Sequences to Stochastic

Processes Valérie Girardin

https://textbookfull.com/product/applied-probability-from-random-
sequences-to-stochastic-processes-valerie-girardin/

Fundamentals of Probability With Stochastic Processes

Solution Manual 4th Edition Ghahramani Saeed

https://textbookfull.com/product/fundamentals-of-probability-
with-stochastic-processes-solution-manual-4th-edition-ghahramani-
saeed/

Theory and Statistical Applications of Stochastic

Processes 1st Edition Yuliya Mishura

https://textbookfull.com/product/theory-and-statistical-
applications-of-stochastic-processes-1st-edition-yuliya-mishura/

Introduction to Stochastic Processes 1st Edition Tapas

Kumar Chandra

https://textbookfull.com/product/introduction-to-stochastic-
processes-1st-edition-tapas-kumar-chandra/
 Theory of
Stochastic Objects
Probability, Stochastic Processes
and Inference
CHAPMAN & HALL/CRC
Texts in Statistical Science Series
Series Editors
Joseph K. Blitzstein, Harvard University, USA
Julian J. Faraway, University of Bath, UK
Martin Tanner, Northwestern University, USA
Jim Zidek, University of British Columbia, Canada
Statistical Theory: A Concise Introduction
F. Abramovich and Y. Ritov
Practical Multivariate Analysis, Fifth Edition
A. Afifi, S. May, and V.A. Clark
Practical Statistics for Medical Research
D.G. Altman
Interpreting Data: A First Course
in Statistics
A.J.B. Anderson
Introduction to Probability with R
K. Baclawski
Linear Algebra and Matrix Analysis for Statistics
S. Banerjee and A. Roy
Modern Data Science with R
B. S. Baumer, D. T. Kaplan, and N. J. Horton
Mathematical Statistics: Basic Ideas and Selected
Topics, Volume I, Second Edition
P. J. Bickel and K. A. Doksum
Mathematical Statistics: Basic Ideas and Selected
Topics, Volume II
P. J. Bickel and K. A. Doksum
Analysis of Categorical Data with R
C. R. Bilder and T. M. Loughin
Statistical Methods for SPC and TQM
D. Bissell
Introduction to Probability
J. K. Blitzstein and J. Hwang
Bayesian Methods for Data Analysis, Third Edition
B.P. Carlin and T.A. Louis
Statistics in Research and Development,
Second Edition
R. Caulcutt
The Analysis of Time Series: An Introduction,
Sixth Edition
C. Chatfield
Introduction to Multivariate Analysis
C. Chatfield and A.J. Collins
Problem Solving: A Statistician’s Guide,
Second Edition
C. Chatfield
Statistics for Technology: A Course in Applied
Statistics, Third Edition
C. Chatfield
Analysis of Variance, Design, and Regression:
Linear Modeling for Unbalanced Data,
Second Edition
R. Christensen
Bayesian Ideas and Data Analysis: An Introduction
for Scientists and Statisticians
R. Christensen, W. Johnson, A. Branscum,
and T.E. Hanson
Modelling Binary Data, Second Edition
D. Collett
Modelling Survival Data in Medical Research, Third
Edition
D. Collett
Introduction to Statistical Methods for
Clinical Trials
T.D. Cook and D.L. DeMets
Applied Statistics: Principles and Examples
D.R. Cox and E.J. Snell
Multivariate Survival Analysis and Competing Risks
M. Crowder
Statistical Analysis of Reliability Data
M.J. Crowder, A.C. Kimber, T.J. Sweeting,
and R.L. Smith
An Introduction to Generalized Linear Models,
Third Edition
A.J. Dobson and A.G. Barnett
Nonlinear Time Series: Theory, Methods, and
Applications with R Examples
R. Douc, E. Moulines, and D.S. Stoffer
Introduction to Optimization Methods and Their
Applications in Statistics
B.S. Everitt
Extending the Linear Model with R: Generalized
Linear, Mixed Effects and Nonparametric Regression
Models, Second Edition
J.J. Faraway
Linear Models with R, Second Edition
J.J. Faraway
A Course in Large Sample Theory
T.S. Ferguson
Multivariate Statistics: A Practical Approach
B. Flury and H. Riedwyl
Readings in Decision Analysis Exercises and Solutions in Statistical Theory
S. French L.L. Kupper, B.H. Neelon, and S.M. O’Brien
Discrete Data Analysis with R: Visualization and Design and Analysis of Experiments with R
Modeling Techniques for Categorical and Count J. Lawson
Data Design and Analysis of Experiments with SAS
M. Friendly and D. Meyer J. Lawson
Markov Chain Monte Carlo: Stochastic Simulation A Course in Categorical Data Analysis
for Bayesian Inference, Second Edition T. Leonard
D. Gamerman and H.F. Lopes
Statistics for Accountants
Bayesian Data Analysis, Third Edition S. Letchford
A. Gelman, J.B. Carlin, H.S. Stern, D.B. Dunson,
A. Vehtari, and D.B. Rubin Introduction to the Theory of Statistical Inference
H. Liero and S. Zwanzig
Multivariate Analysis of Variance and Repeated
Measures: A Practical Approach for Behavioural Statistical Theory, Fourth Edition
Scientists B.W. Lindgren
D.J. Hand and C.C. Taylor Stationary Stochastic Processes: Theory and
Practical Longitudinal Data Analysis Applications
D.J. Hand and M. Crowder G. Lindgren

Linear Models and the Relevant Distributions and Statistics for Finance
Matrix Algebra E. Lindström, H. Madsen, and J. N. Nielsen
D.A. Harville The BUGS Book: A Practical Introduction to
Logistic Regression Models Bayesian Analysis
J.M. Hilbe D. Lunn, C. Jackson, N. Best, A. Thomas, and
D. Spiegelhalter
Richly Parameterized Linear Models: Additive, Time
Series, and Spatial Models Using Random Effects Introduction to General and Generalized
J.S. Hodges Linear Models
H. Madsen and P. Thyregod
Statistics for Epidemiology
N.P. Jewell Time Series Analysis
H. Madsen
Stochastic Processes: An Introduction, Third Edition
P.W. Jones and P. Smith Pólya Urn Models
H. Mahmoud
The Theory of Linear Models
B. Jørgensen Randomization, Bootstrap and Monte Carlo
Methods in Biology, Third Edition
Pragmatics of Uncertainty B.F.J. Manly
J.B. Kadane
Statistical Regression and Classification:
Principles of Uncertainty From Linear Models to Machine Learning
J.B. Kadane N. Matloff
Graphics for Statistics and Data Analysis with R Introduction to Randomized Controlled Clinical
K.J. Keen Trials, Second Edition
Mathematical Statistics J.N.S. Matthews
K. Knight Statistical Rethinking: A Bayesian Course with
Introduction to Functional Data Analysis Examples in R and Stan
P. Kokoszka and M. Reimherr R. McElreath
Introduction to Multivariate Analysis: Linear and Statistical Methods in Agriculture and Experimental
Nonlinear Modeling Biology, Second Edition
S. Konishi R. Mead, R.N. Curnow, and A.M. Hasted
Nonparametric Methods in Statistics with SAS Statistics in Engineering: A Practical Approach
Applications A.V. Metcalfe
O. Korosteleva Theory of Stochastic Objects: Probability, Stochastic
Modeling and Analysis of Stochastic Systems, Processes and Inference
Third Edition A.C. Micheas
V.G. Kulkarni Statistical Inference: An Integrated Approach,
Exercises and Solutions in Biostatistical Theory Second Edition
L.L. Kupper, B.H. Neelon, and S.M. O’Brien H. S. Migon, D. Gamerman, and F. Louzada
Beyond ANOVA: Basics of Applied Statistics Spatio-Temporal Methods in Environmental
R.G. Miller, Jr. Epidemiology
G. Shaddick and J.V. Zidek
A Primer on Linear Models
J.F. Monahan Decision Analysis: A Bayesian Approach
J.Q. Smith
Stochastic Processes: From Applications to Theory
P.D Moral and S. Penev Analysis of Failure and Survival Data
P. J. Smith
Applied Stochastic Modelling, Second Edition
B.J.T. Morgan Applied Statistics: Handbook of GENSTAT
Analyses
Elements of Simulation
B.J.T. Morgan E.J. Snell and H. Simpson

Probability: Methods and Measurement Applied Nonparametric Statistical Methods,

A. O’Hagan Fourth Edition
P. Sprent and N.C. Smeeton
Introduction to Statistical Limit Theory
A.M. Polansky Data Driven Statistical Methods
P. Sprent
Applied Bayesian Forecasting and Time Series
Analysis Generalized Linear Mixed Models:
A. Pole, M. West, and J. Harrison Modern Concepts, Methods and Applications
W. W. Stroup
Statistics in Research and Development,
Time Series: Modeling, Computation, and Inference Survival Analysis Using S: Analysis of
R. Prado and M. West Time-to-Event Data
M. Tableman and J.S. Kim
Essentials of Probability Theory for Statisticians
M.A. Proschan and P.A. Shaw Applied Categorical and Count Data Analysis
W. Tang, H. He, and X.M. Tu
Introduction to Statistical Process Control
P. Qiu Elementary Applications of Probability Theory,
Second Edition
Sampling Methodologies with Applications H.C. Tuckwell
P.S.R.S. Rao
Introduction to Statistical Inference and Its
A First Course in Linear Model Theory Applications with R
N. Ravishanker and D.K. Dey M.W. Trosset
Essential Statistics, Fourth Edition Understanding Advanced Statistical Methods
D.A.G. Rees P.H. Westfall and K.S.S. Henning
Stochastic Modeling and Mathematical Statistics: Statistical Process Control: Theory and Practice,
A Text for Statisticians and Quantitative Scientists Third Edition
F.J. Samaniego G.B. Wetherill and D.W. Brown
Statistical Methods for Spatial Data Analysis Generalized Additive Models: An Introduction
O. Schabenberger and C.A. Gotway with R, Second Edition
Bayesian Networks: With Examples in R S. Wood
M. Scutari and J.-B. Denis Epidemiology: Study Design and
Large Sample Methods in Statistics Data Analysis, Third Edition
P.K. Sen and J. da Motta Singer M. Woodward
Introduction to Statistical Methods for Financial Practical Data Analysis for Designed Experiments
Models B.S. Yandell
T. A. Severini
 Texts in Statistical Science

Theory of
Stochastic Objects
Probability, Stochastic Processes
and Inference

Athanasios Christou Micheas

Department of Statistics, University of Missouri, USA
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2018 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Printed on acid-free paper
Version Date: 20171219
International Standard Book Number-13: 978-1-4665-1520-8 (Hardback)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been
made to publish reliable data and information, but the author and publisher cannot assume responsibility for the
validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the
copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to
publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let
us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or
utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including
photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission
from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.com
(http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA
01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For
organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for
identification and explanation without intent to infringe.

Library of Congress Cataloging-in-Publication Data

Names: Micheas, Athanasios Christou, author.

Title: Theory of stochastic objects : probability, stochastic processes and
inference / by Athanasios Christou Micheas.
Description: Boca Raton, Florida : CRC Press, [2018] | Includes
bibliographical references and index.
Identifiers: LCCN 2017043053| ISBN 9781466515208 (hardback) | ISBN
9781315156705 (e-book)
Subjects: LCSH: Point processes. | Stochastic processes.
Classification: LCC QA274.42 .M53 2018 | DDC 519.2/3--dc23
LC record available at https://lccn.loc.gov/2017043053

Visit the Taylor & Francis Web site at

http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
To my family
 Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi

List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii

List of Abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv

List of Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii

List of Distribution Notations . . . . . . . . . . . . . . . . . . . . . . xxix

1 Rudimentary Models and Simulation Methods . . . . . . . . . . . 1

1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Rudimentary Probability . . . . . . . . . . . . . . . . . . . . . . 2
1.2.1 Probability Distributions . . . . . . . . . . . . . . . . . . . 4
1.2.2 Expectation . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2.3 Mixtures of Distributions . . . . . . . . . . . . . . . . . . 9
1.2.4 Transformations of Random Vectors . . . . . . . . . . . . . 11
1.3 The Bayesian Approach . . . . . . . . . . . . . . . . . . . . . . 11
1.3.1 Conjugacy . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.3.2 General Prior Selection Methods and Properties . . . . . . 14
1.3.3 Hierarchical Bayesian Models . . . . . . . . . . . . . . . . 15
1.4 Simulation Methods . . . . . . . . . . . . . . . . . . . . . . . . 17
1.4.1 The Inverse Transform Algorithm . . . . . . . . . . . . . . 17
1.4.2 The Acceptance-Rejection Algorithm . . . . . . . . . . . . 18
1.4.3 The Composition Method for Generating Mixtures . . . . . 20
1.4.4 Generating Multivariate Normal and Related Distributions . 20
1.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
1.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2 Statistical Inference . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.2 Decision Theory . . . . . . . . . . . . . . . . . . . . . . . . . . 27
ix
x CONTENTS
2.3 Point Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.3.1 Classical Methods . . . . . . . . . . . . . . . . . . . . . . 30
2.3.2 Bayesian Approach . . . . . . . . . . . . . . . . . . . . . . 39
2.3.3 Evaluating Point Estimators Using Decision Theory . . . . 40
2.3.4 Convergence Concepts and Asymptotic Behavior . . . . . . 42
2.4 Interval Estimation . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.4.1 Confidence Intervals . . . . . . . . . . . . . . . . . . . . . 45
2.4.2 Highest Posterior Density Credible Sets . . . . . . . . . . . 46
2.4.3 Decision Theoretic . . . . . . . . . . . . . . . . . . . . . . 48
2.5 Hypothesis Testing . . . . . . . . . . . . . . . . . . . . . . . . . 49
2.5.1 Classic Methods . . . . . . . . . . . . . . . . . . . . . . . 49
2.5.2 Bayesian Testing Procedures . . . . . . . . . . . . . . . . . 55
2.5.3 Decision Theoretic . . . . . . . . . . . . . . . . . . . . . . 58
2.5.4 Classical and Bayesian p-values . . . . . . . . . . . . . . . 60
2.5.5 Reconciling the Bayesian and Classical Paradigms . . . . . 65
2.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
2.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

3 Measure and Integration Theory . . . . . . . . . . . . . . . . . . . 77

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.2 Deterministic Set Theory . . . . . . . . . . . . . . . . . . . . . . 77
3.3 Topological Spaces and σ-fields . . . . . . . . . . . . . . . . . . 79
3.4 Product Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
3.5 Measurable Spaces and Mappings . . . . . . . . . . . . . . . . . 86
3.6 Measure Theory and Measure Spaces . . . . . . . . . . . . . . . 91
3.6.1 Signed Measures and Decomposition Theorems . . . . . . 97
3.6.2 Carathéodory Measurability and Extension Theorem . . . . 100
3.6.3 Construction of the Lebesgue Measure . . . . . . . . . . . 103
3.7 Defining Integrals with Respect to Measures . . . . . . . . . . . . 106
3.7.1 Change of Variable and Integration over Sets . . . . . . . . 113
3.7.2 Lebesgue, Riemann and Riemann-Stieltjes Integrals . . . . 115
3.7.3 Radon-Nikodym Theorem . . . . . . . . . . . . . . . . . . 118
3.7.4 Product Measure and Fubini Theorem . . . . . . . . . . . . 121
3.7.5 L p -spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
3.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
3.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

4 Probability Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
4.2 Probability Measures and Probability Spaces . . . . . . . . . . . 139
4.2.1 Extension of Probability Measure . . . . . . . . . . . . . . 140
4.2.2 Defining Random Objects . . . . . . . . . . . . . . . . . . 143
4.2.3 Distribution Functions and Densities . . . . . . . . . . . . 148
CONTENTS xi
4.2.4 Independence . . . . . . . . . . . . . . . . . . . . . . . . . 153
4.2.5 Calculating Probabilities for Limits of Events . . . . . . . . 158
4.2.6 Expectation of a Random Object . . . . . . . . . . . . . . . 159
4.2.7 Characteristic Functions . . . . . . . . . . . . . . . . . . . 162
4.3 Conditional Probability . . . . . . . . . . . . . . . . . . . . . . . 165
4.3.1 Conditioning on the Value of a Random Variable . . . . . . 165
4.3.2 Conditional Probability and Expectation Given a σ-field . . 170
4.3.3 Conditional Independence Given a σ-field . . . . . . . . . 177
4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178
4.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

5 Convergence of Random Objects . . . . . . . . . . . . . . . . . . 183

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
5.2 Existence of Independent Sequences of Random Variables . . . . 183
5.3 Limiting Behavior of Sequences of Random Variables . . . . . . 184
5.3.1 Slutsky and Cramér Theorems . . . . . . . . . . . . . . . . 184
5.3.2 Consistency of the MLE . . . . . . . . . . . . . . . . . . . 186
5.4 Limiting Behavior of Probability Measures . . . . . . . . . . . . 187
5.4.1 Integrating Probability Measures to the Limit . . . . . . . . 187
5.4.2 Compactness of the Space of Distribution Functions . . . . 191
5.4.3 Weak Convergence via Non-Central Moments . . . . . . . 193
5.5 Random Series . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
5.5.1 Convolutions . . . . . . . . . . . . . . . . . . . . . . . . . 194
5.5.2 Fourier Inversion and the Continuity Theorem . . . . . . . 196
5.5.3 Limiting Behavior of Partial Sums . . . . . . . . . . . . . . 199
5.5.4 Central Limit Theorems . . . . . . . . . . . . . . . . . . . 200
5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
5.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204

6 Random Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . 207

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
6.2 Definitions and Properties . . . . . . . . . . . . . . . . . . . . . 207
6.3 Martingales . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
6.3.1 Filtrations and Stopping Times . . . . . . . . . . . . . . . 215
6.3.2 Convergence Theorems . . . . . . . . . . . . . . . . . . . 216
6.3.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . 223
6.3.4 Wald Identities and Random Walks . . . . . . . . . . . . . 225
6.4 Renewal Sequences . . . . . . . . . . . . . . . . . . . . . . . . . 226
6.5 Markov Chains . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
6.5.1 Definitions and Properties . . . . . . . . . . . . . . . . . . 233
6.5.2 Discrete State Space . . . . . . . . . . . . . . . . . . . . . 236
6.5.3 The Martingale Problem . . . . . . . . . . . . . . . . . . . 240
6.5.4 Visits to Fixed States: General State Space . . . . . . . . . 241
xii CONTENTS
6.5.5 Visits to Fixed States: Discrete State Space . . . . . . . . . 243
6.5.6 State Classification . . . . . . . . . . . . . . . . . . . . . . 248
6.5.7 Limiting Behavior . . . . . . . . . . . . . . . . . . . . . . 255
6.6 Stationary Sequences and Ergodicity . . . . . . . . . . . . . . . . 258
6.7 Applications to Markov Chain Monte Carlo Methods . . . . . . . 262
6.7.1 Metropolis-Hastings Algorithm . . . . . . . . . . . . . . . 263
6.7.2 Gibbs Sampling . . . . . . . . . . . . . . . . . . . . . . . 264
6.7.3 Reversible Jump Markov Chain Monte Carlo . . . . . . . . 265
6.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
6.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269

7 Stochastic Processes . . . . . . . . . . . . . . . . . . . . . . . . . 275

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
7.2 The Poisson Process . . . . . . . . . . . . . . . . . . . . . . . . 277
7.3 General Stochastic Processes . . . . . . . . . . . . . . . . . . . . 279
7.3.1 Continuous Time Filtrations and Stopping Times . . . . . . 281
7.3.2 Continuous Time Martingales . . . . . . . . . . . . . . . . 283
7.3.3 Kolmogorov Existence Theorem . . . . . . . . . . . . . . . 285
7.4 Markov Processes . . . . . . . . . . . . . . . . . . . . . . . . . . 289
7.4.1 Càdlàg Space . . . . . . . . . . . . . . . . . . . . . . . . . 289
7.4.2 Infinitesimal Generators . . . . . . . . . . . . . . . . . . . 294
7.4.3 The Martingale Problem . . . . . . . . . . . . . . . . . . . 296
7.4.4 Construction via Subordinated Markov Chains . . . . . . . 297
7.4.5 Discrete State Space . . . . . . . . . . . . . . . . . . . . . 298
7.4.5.1 Sample Paths and State Classification . . . . . . . . 301
7.4.5.2 Construction via Jump Times . . . . . . . . . . . . 303
7.4.5.3 Infinitesimal Generator and Transition Function . . 305
7.4.5.4 Limiting Behavior . . . . . . . . . . . . . . . . . . 307
7.4.5.5 Birth-Death Processes . . . . . . . . . . . . . . . . 309
7.4.6 Brownian Motion . . . . . . . . . . . . . . . . . . . . . . . 312
7.4.7 Construction of the Wiener Measure . . . . . . . . . . . . . 314
7.5 Building on Martingales and Brownian Motion . . . . . . . . . . 317
7.5.1 Stochastic Integrals . . . . . . . . . . . . . . . . . . . . . . 317
7.5.2 Stochastic Differential Equations . . . . . . . . . . . . . . 324
7.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331
7.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332

A Additional Topics and Complements . . . . . . . . . . . . . . . . 337

p
A.1 Mappings in R . . . . . . . . . . . . . . . . . . . . . . . . . . . 337
A.2 Topological, Measurable and Metric Spaces . . . . . . . . . . . . 338
A.3 Baire Functions and Spaces . . . . . . . . . . . . . . . . . . . . . 342
A.4 Fisher Information . . . . . . . . . . . . . . . . . . . . . . . . . 343
A.5 Multivariate Analysis Topics . . . . . . . . . . . . . . . . . . . . 344
CONTENTS xiii
A.6 State Classification . . . . . . . . . . . . . . . . . . . . . . . . . 345
A.7 MATLAB R Code Function Calls . . . . . . . . . . . . . . . . . 346
A.8 Commonly Used Distributions and Their Densities . . . . . . . . 347

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
 Preface

Random variables and random vectors have been well defined and studied for
over a century. Subsequently, in the history of statistical science, researchers began
considering collections of points together, which gave birth to point process theory
and more recently, to random set theory. This was mainly motivated due to advances
in technology and the types of data that experimenters began investigating, which
in turn led to the creation and investigation of advanced statistical methods able to
handle such data.
In this book we take the reader on a journey through some of the most essential
topics in mathematics and statistics, constantly building on previous concepts, mak-
ing the transition from elementary statistical inference to the advanced probabilistic
treatment more natural and concrete. Our central focus is defining and exploring the
concept of a random quantity or object in different contexts, where depending on
the data under consideration, “random objects” are described using random vari-
ables, vectors or matrices, stochastic processes, integrals and differential equations,
or point processes and random sets.
This view of random objects has not been adequately investigated and pre-
sented in mathematics and statistics textbooks that are out there since they have
mostly concentrated on specific parts of the aforementioned concepts. This is one
of the reasons why I undertake the task of writing a textbook that would present
the knowledge in a concrete way, through examples and exercises, which is sorely
needed in understanding statistical inference, probability theory and stochastic pro-
cesses. This approach will help the instructor of these topics to engage the students
through problem sets and present the theory and applications involved in a way that
they will appreciate.
Since this monumental task cannot be accomplished in a single textbook, the
theoretical and modeling topics considered have been organized in two texts; this
text is concerned with rudimentary to advanced theoretical aspects of random ob-
jects based on random variables, including statistical inference, probability theory
and stochastic processes. The modeling of these objects and their applications to
real life data is presented in the text Theory and Modeling of Stochastic Objects:
Point Processes and Random Sets (forthcoming, hereafter referred to as TMSO-
PPRS). The latter stochastic objects are a natural extension of random variables

xv
xvi PREFACE
and vectors and we can think of the TMSO-PPRS text as a natural continuation of
the theory presented herein.
In particular, we present a comprehensive account of topics in statistics in a
way that can be a natural extension of a more traditional graduate course in prob-
ability theory. This is especially true for Chapters 1 and 2, which is a feature that
has been lacking from available texts in probability theory. Another distinguishing
feature of this text is that we have included an amazing amount of material. More
precisely, one would need to use at least one book on real analysis, one in measure
and/or probability theory, one in stochastic processes, and at least one on statistics
to capture just the expository material that has gone into this text.
Being a teacher and mentor to undergraduate and graduate students, I have seen
their attempts to comprehend new material from rudimentary to advanced mathe-
matical and statistical concepts. I have also witnessed their struggles with essential
topics in statistics, such as defining a probability space for a random variable, which
is one of the most important constructs in statistics. This book attempts to introduce
these concepts in a novel way making it more accessible to students and researchers
through examples. This approach is lacking in most textbooks/monographs that one
can use to teach students.
Instructors and researchers in academia often find themselves complementing
material from several books in order to provide a spherical overview of the topics
of a class. This book is the result of my efforts over the years to provide comprehen-
sive and compact accounts of topics I had to teach to undergraduate and graduate
students.
Therefore, the book is targeted toward students at the master’s and Ph.D. levels,
as well as academicians in the mathematics and statistics disciplines. Although the
concepts will be built from the master’s level up, the book addresses advanced read-
ers in the later chapters. When used as a textbook, prior knowledge of probability
or measure theory is welcomed but not necessary.
In particular, Chapters 1 and 2 can be used for several courses on statistical
inference with minor additions for any proofs the instructor chooses to further il-
lustrate. In these chapters we summarize over a century and a half of development
in mathematical statistics. Depending on the level of the course, the instructor can
select specific exercises to supplement the text, in order to provide a better under-
standing and more depth into the concepts under consideration. For example, using
selectively the material and exercises from Chapters 1, 2, 4 and 5, I have taught
several sequences on statistical inference at the University of Missouri (MU), in-
cluding Stat7750/60 and 4750/60 (statistical inference course at the undergraduate
and master’s level), Stat8710/20 (intermediate statistical inference course at the
Ph.D. level) and Stat9710/20 (advanced inference for Ph.D. students).
At the master’s level, it is recommended that the instructor omits advanced top-
ics from Chapter 2, including most of the decision-theoretic topics and the corre-
PREFACE xvii
sponding proofs of the relevant results. Basic theorems and their proofs, such as
the Bayes or the factorization theorem, should be presented to the students in de-
tail. The proofs of such results are included as exercises, and the instructor can use
the solution manual in order to choose what they deem appropriate to illustrate to
the students.
For example, when teaching a statistical inference course for Ph.D. students, all
concepts presented in Chapters 1 and 2 should be introduced, as well as topics on
asymptotics from Chapter 5. However, certain proofs might be beyond the level of
an intermediate statistical inference course for Ph.D. students. For example, when
it comes to introducing evaluation of point estimators, we may omit the explicit
proof of all parts of the important remark 2.12 and simply present the material, or
the compactness results in Chapter 5, and focus only on the central limit theorems
or Slutsky and Cramér theorems.
For an advanced course on statistical inference at the Ph.D. level, one would
omit most of the rudimentary results of Chapter 1, and focus on topics from Chap-
ter 2 (inference), Chapter 4 (e.g., characteristic functions), and Chapter 5 (asymp-
totics), including all the important proofs of the theorems and remarks presented in
the text. Once again, the instructor can find the solution manual invaluable in this
case, since it will allow them to select the topics they want to present along with
concrete proofs.
Chapters 3-5 can be used to introduce measure theoretic probability to mathe-
matics and statistics graduate students. Some of the proofs should be skipped since
it would take more than one semester to go through all the material. More precisely,
over the past decade when I taught the advanced probability theory course Stat9810
at MU, I had to omit most of the measure theoretic proofs and be quite selective in
the material for a one-semester course. For example, important theorems and their
proofs, like Fubini, Kolmogorov 0-1 Law, Radon-Nikodym or Kolmogorov Three
Series, should be illustrated to the students in detail.
In contrast, one may skip the proofs of the theoretical development of the
Carathodory extension theorem, or omit the proofs of the decomposition theorems
(Chapter 3) and the compactness theorems of Chapter 5. Of course, most of the
important results in measure and probability theory and their proofs are still there
for the inquisitive student and researcher who needs to go deeper. These chapters
are fairly comprehensive and self-contained, which is important for Ph.D. students
that have not had an advanced real analysis course.
Chapter 6 is a fairly comprehensive account of stochastic processes in discrete
time and in particular Markov chains. This material has been used to teach an intro-
ductory course on stochastic processes to both undergraduate and master’s students
(Stat4850/7850), as well as Ph.D.-level students in one semester (Stat 9820, a con-
tinuation of Stat9810). Note that most of the development and exposition of discrete
Markov chains and processes does not require heavy measure theory as presented
xviii PREFACE
in Chapters 6 and 7, therefore making it accessible to a wide variety of students, in-
cluding undergraduates. A good working knowledge of matrix algebra is required
in this case, which is a requirement for the undergraduate and graduate students
when they take this course. In particular, the instructor simply needs to explain in a
rudimentary way “transition probability measures,” e.g., replace it with the notion
of transition probabilities and matrices, and then the material can be presented to
the students in a non-measure theoretic way.
The material in Chapter 7 has been used to teach stochastic processes in con-
tinuous time to Ph.D. (Stat 9820) and advanced master’s level students, including
topics from Chapter 6, as mentioned above. The instructor can supplement materi-
als from other chapters as they see fit in order to build the mathematical foundations
of the concepts presented as needed. For example, in the beginning of the class we
may conduct a mini review of probability theory and Markov chains before jumping
into continuous time stochastic processes.
As you begin reading, several features that help with the learning process should
immediately draw your attention; each chapter begins with basic illustrations and
ends with a more advanced treatment of the topic at hand. We are exploring and
reconciling, when feasible, both the frequentist and Bayesian approaches to the
topics considered. In addition, recent developments in statistics are presented or
referenced in the text and summary of each chapter.
Proofs for most of the theorems, lemmas and remarks presented in each chapter
are given in the text or are requested as exercises, with the exception of the rudi-
mentary Chapters 1 and 2, where the proofs are requested as exercises only. Proofs
and additional information on the topics discussed can be found in the books or
journal papers referenced at the summary section of each chapter. Of course, the
interested reader can find proofs to selected exercises in the supplementary online
material for the book (see website below).
The theorems and results presented in the text can range from easy to compli-
cated, and therefore, we usually follow them with an illustrative remark or example
to explain the new concept. To further help in our understanding of the material
and for quick reference, various topics and complements from mathematics and
statistics are included in an appendix.
The MATLAB R code used for the examples presented along with solutions
to exercises and other material, such as errata, can be found at the book website
https://www.crcpress.com/9781466515208.

There are many people that have contributed, in their own way, to the creation
of this book. I am grateful to the faculty members of the Department of Statistics at
the University of Missouri, USA, for their constructive interactions and discussions
over the years. In particular, special thanks go to my friends and colleagues Christo-
pher Wikle, Scott Holan, Stamatis Dostoglou and Joe Cavanaugh (University of
PREFACE xix
Iowa), and my friend and mentor Konstantinos Zografos from the Department of
Mathematics, University of Ioannina, Greece. Lastly, my academic advisor, Distin-
guished Professor of Statistics Dipak Dey, Department of Statistics, University of
Connecticut, USA, has been an inspiration to me over the years.
I am grateful to Professors Stamatis Dostoglou, Department of Mathematics,
University of Missouri, USA, Georg Lindgren, Department of Mathematical Statis-
tics, Centre for Mathematical Sciences, Lund, Sweden, and an anonymous re-
viewer, for their invaluable comments and suggestions regarding earlier versions
of the manuscript. Special thanks go to my friend and colleague Distinguished Pro-
fessor Noel Cressie, School of Mathematics and Applied Statistics, University of
Wollongong, Australia, for his support and encouragement over the years as I was
working on the manuscript, as well as for his advice regarding all aspects of the
book, including its title.
Additional thanks go to the hundreds of students for their undivided attention
while they had to take classes from me on these topics and have helped me better
myself through the teaching process. In particular, special thanks goes to all my
graduate students, especially to Jiaxun Chen and Alex Oard. I am also grateful to
Rob Calver, Michele Dimont, Becky Condit and the friends at Chapman-Hall/CRC
for their patience while the manuscript was composed and for their help with the
copy edit process.
Above all, my appreciation and love to my family, my daughters Vaso, Evi and
Christina, my wife Lada and my father Christos, for their unconditional love and
understanding.
I apologize in advance for any typos or errors in the text and I would be grateful
for any comments, suggestions or corrections the kind reader would like to bring to
my attention.
Sakis Micheas
December 2017
 List of Figures

4.1 Displaying random objects: (a) random functions, (b)-(d) random

counting measures showing regularity, randomness and clustering
of the points, (e) random discs, and (f) Gaussian random field. . . 149

7.1 Brownian motion realizations: (a) univariate, (b) bivariate. . . . . 314

xxi
 List of Tables

2.1 Schematic for any hypothesis testing problem along with the
occurrence of the Type I and II errors. . . . . . . . . . . . . . . . 50
2.2 Simulations of Monte Carlo goodness-of-fit tests. We used
L = 100000 predictive samples for n = 10, 50, 100 observed sample
points and the data is simulated from three models Uni f (0, 1),
Gamma(10, 10), and N(−10, 1). We choose λ = 1 for the
hyperparameter, and p pred is provided for four statistics
T 1 (X) = X(1) , T 2 (X) = X(n) , T 3 (X) = X, and T 4 (X) = S 2 . Based on
these results T 1 emerges as the best test statistic in order to assess
the entertained model. . . . . . . . . . . . . . . . . . . . . . . . 64

4.1 Characteristic functions of commonly used continuous

distributions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
4.2 Characteristic functions of commonly used discrete distributions. 166

xxiii
 List of Abbreviations

a.s. Almost Surely

BCT Bounded Convergence Theorem
Be Beta function
Càdlàg Continue à Droite, Limites à Gauche (RCLL)
cdf Cumulative Distribution Function
cf Characteristic Function
CI Confidence Interval
CLT Central Limit Theorem
CR-LB Cramer-Rao Lower Bound
DCT Dominated Convergence Theorem
ev. Eventually
HBM Hierarchical Bayesian Model
HPD Highest Posterior Density
iid Independent and Identically Distributed
i.o. Infinitely Often
LHS Left-Hand Side
MAP Maximum a Posteriori
MCMC Markov Chain Monte Carlo
MCT Monotone Convergence Theorem
MLE Maximum Likelihood Estimator
MG Martingale
mgf Moment Generating Function
PEL Posterior Expected Loss
pgf Probability Generating Function

xxv
xxvi LIST OF ABBREVIATIONS
PP Point Process
RHS Right-Hand Side
RCLL Right Continuous Left Limits
SLLN Strong Law of Large Numbers
sMG Sub-Martingale
SMG Super-Martingale
TMSO-PPRS Theory and Modeling of Stochastic Objects: Point Processes
and Random Sets
UMVUE Uniformly Minimum Variance Unbiased Estimator
WLLN Weak Law of Large Numbers
wlog Without Loss of Generality
w.p. With Probability
 List of Symbols

Q≪µ Q is absolutely continuous with respect to µ

B(X) Borel sets of the space X
DΨ[0,+∞) Càdlàg space: Ψ-valued functions defined on
[0, +∞) with RCLL
M or M(µ∗ ) Carathéodory measurable sets
n!
C nx = (n−x)!x!
Combination: the number of ways we can select x
objects out of n objects
n! P
k
C nx1 ,x2 ,...,xk = x1 !x2 !...xk !
Multinomial Coefficient, xi = n,
i=1
xi = 0, 1, . . . , n, i = 1, 2, . . . , k
CR[0,+∞) Continuous R-valued functions defined on [0, +∞)
↓ Decreasing Sequence Converging to
d
X=Y X and Y have the same distribution
X ∼ fX , F X , QX X has density fX or cdf F X or distribution QX
x! = 1 ∗ 2 ∗ . . . ∗ (x − 1) ∗ x Factorial of an integer x
F Closed subsets of R p
∀ For every (or for all)
∃ There exist(s)
⇒ Implies
⇔ If and only if
↑ Increasing Sequence Converging to
I(x ∈ A) = IA (x) Indicator function of the set A, 1 if x ∈ A, 0 if x < A
µp Lebesgue measure on R p
ω1 ∨ ω2 max{ω1 , ω2 }
ω1 ∧ ω2 min{ω1 , ω2 }

xxvii
xxviii LIST OF SYMBOLS
an
an = o(bn ) bn
→ 0 as n → ∞
1:1 One-to-one (function)
h dQ i
dµ
Radon-Nikodym derivative of Q with respect to µ
p
R, R Real numbers in 1 and p dimensions
R = R ∪ {−∞} ∪ {+∞} Extended real line
R+ , R0+ {x ∈ R : x > 0}, {x ∈ R : x ≥ 0}
Q⊥µ Q and µ are mutually singular measures
[P], [µ] With respect to measure P or µ
Z Integers, {. . . , −2, −1, 0, 1, 2, . . .}
Z = Z ∪ {−∞} ∪ {+∞} Extended integers
Z+ , Z+0 , Z+ {1, 2, . . .}, {0, 1, 2, . . .}, {0, 1, 2, . . . , +∞}
Another random document with
no related content on Scribd:
A beautiful thread, than a thread of sin.

THE FORTUNATE ISLES

By Joaquin Miller

You sail and you seek for the Fortunate Isles,

The old Greek Isles of the yellow bird’s song?
Then steer straight on through the watery miles,
Straight on, straight on and you can’t go wrong.
Nay not to the left, nay not to the right,
But on, straight on, and the Isles are in sight,
The Fortunate Isles where the yellow birds sing
And life lies girt with a golden ring.

These Fortunate Isles they are not so far,

They lie within reach of the lowliest door;
You can see them gleam by the twilight star;
You can hear them sing by the moon’s white shore—
Nay, never look back! Those leveled grave stones
They were landing steps; they were steps unto thrones
Of glory for souls that have sailed before,
And have set white feet on the fortunate shore.

And what are the names of the Fortunate Isles?

Why, Duty and Love and a large Content.
Lo! these are the Isles of the watery miles,
That God let down from the firmament.
Lo! Duty, and Love, and a true man’s Trust;
Your forehead to God though your feet in the dust;
Lo! Duty, and Love, and a sweet Babe’s Smiles,
And these, O friend, are the Fortunate Isles.

—Copyright by Harr Wagner Co., San Francisco, and used by kind

permission of author and publisher.

YOSEMITE
 By Joaquin Miller

Sound! sound! sound!

O colossal walls and crown’d
In one eternal thunder!
Sound! sound! sound!
O ye oceans overhead,
While we walk, subdued in wonder,
In the ferns and grasses, under
And beside the swift Merced!

Fret! fret! fret!

Streaming, sounding banners, set
On the giant granite castles
In the clouds and in the snow!
But the foe he comes not yet,—
We are loyal, valiant vassals,
And we touch the trailing tassels
Of the banners far below.

Surge! surge! surge!

From the white Sierra’s verge,
To the very valley blossom.
Surge! surge! surge!
Yet the song-bird builds a home,
And the mossy branches cross them,
And the tasseled tree-tops toss them,
In the clouds of falling foam.

Sweep! sweep! sweep!

O ye heaven-born and deep,
In one dread, unbroken chorus!
We may wonder or may weep,—
We may wait on God before us;
We may shout or lift a hand,—
We may bow down and deplore us,
But may never understand.
 Beat! beat! beat!
We advance, but would retreat
From this restless, broken breast
Of the earth in a convulsion.
We would rest, but dare not rest,
For the angel of expulsion
From this Paradise below
Waves us onward and ... we go.

—Copyright by Harr Wagner Co., San Francisco, and used by kind

permission of author and publisher.

THE DEAD MILLIONAIRE

By Joaquin Miller

The gold that with the sunlight lies

In bursting heaps at dawn,
The silver spilling from the skies
At night to walk upon,
The diamonds gleaming in the dew
He never saw, he never knew.

He got some gold, dug from the mud,

Some silver, crushed from stones.
The gold was red with dead man’s blood,
The silver black with groans;
And when he died he moaned aloud,
“There’ll be no pocket in my shroud.”

—Copyright by Harr Wagner Co., San Francisco, and used by kind

permission of author and publisher.

PETER COOPER
(Died 1883)
By Joaquin Miller
Give honor and love forevermore
To this great man gone to rest;
Peace on the dim Plutonian shore,
Rest in the land of the blest.

I reckon him greater than any man

That ever drew sword in war;
I reckon him nobler than king or khan,
Braver and better by far.

And wisest he in this whole wide land

Of hoarding till bent and gray;
For all you can hold in your cold dead hand
Is what you have given away.

So whether to wander the stars or to rest

Forever hushed and dumb,
He gave with a zest and he gave his best—
Give him the best to come.

—Copyright by Harr Wagner Co., San Francisco, and used by kind

permission of author and publisher.

THE VOICE OF THE DOVE

By Joaquin Miller

Come, listen, O Love, to the voice of the dove,

Come, hearken and hear him say:
“There are many To-morrows, my Love, my Love,
There is only one To-day.”

And all day long you can hear him say

This day in purple is rolled
And the baby stars of the milky-way
They are cradled in cradles of gold.

Now what is thy secret serene, gray dove,

 Of singing so sweetly alway?
“There are many To-morrows, my Love, my Love,
There is only one To-day.”

—Copyright by Harr Wagner Co., San Francisco, and used by kind

permission of author and publisher.

WHERE THE WEST BEGINS

By Arthur Chapman

Out where the handclasp’s a little stronger,

Out where a smile dwells a little longer,
That’s where the West begins.
Out where the sun’s a little brighter,
Where the snow that falls is a trifle whiter,
Where the bonds of home are a wee bit tighter,
That’s where the West begins.

Out where the skies are a trifle bluer,

Out where friendship’s a little truer,
That’s where the West begins.
Out where a fresher breeze is blowing,
Where there is laughter in each streamlet flowing,
Where there’s more of reaping and less of sowing,
That’s where the West begins.

Out where the world is in the making,

Where fewer hearts with despair are aching,
That’s where the West begins.
Where there is more of singing and less of sighing,
Where there is more of giving and less of buying,
And a man makes friends without half trying—
That’s where the West begins.

AS I CAME DOWN FROM LEBANON

 By Clinton Scollard

As I came down from Lebanon,

Came winding, wandering slowly down
Through mountain passes bleak and brown,
The cloudless day was well nigh done.
The city like an opal set
In emerald, showed each minaret
Afire with radiant beams of sun,
And glistened orange, fig, and lime,
Where song-birds made melodious chime,
As I came down from Lebanon.

As I came down from Lebanon,

Like lava in the dying glow,
Through olive orchards far below
I saw the murmuring river run;
And ’neath the wall upon the sand
Swart sheiks from distant Samarcand,
With precious spices they had won,
Lay long and languidly in wait
Till they might pass the guarded gate,
As I came down from Lebanon.

As I came down from Lebanon,

I saw strange men from lands afar,
In mosque and square and gay bazar,
The magi that the Moslem shun,
And grave effendi from Stamboul,
Who sherbet sipped in corners cool;
And, from the balconies o’errun
With roses, gleamed the eyes of those
Who dwell in still seraglios,
As I came down from Lebanon.

As I came down from Lebanon,

The flaming flower of daytime died,
And Night, arrayed as is a bride
Of some great king, in garment spun
Of purple and the finest gold,
Outbloomed in glories manifold,
Until the moon, above the dun
And darkening desert, void of shade,
Shone like a keen Damascus blade,
As I came down from Lebanon.

APPLE BLOSSOMS
By William Wesley Martin

Have you seen an apple orchard in the spring? in the spring?

An English apple orchard in the spring?
When the spreading trees are hoary
With their wealth of promised glory,
And the mavis pipes his story
In the spring?

Have you plucked the apple blossoms in the spring? in the spring?
And caught their subtle odors in the spring?
Pink buds bursting at the light,
Crumpled petals baby-white,
Just to touch them a delight!
In the spring!

Have you walked beneath the blossoms in the spring? in the spring?
Beneath the apple blossoms in the spring?
When the pink cascades were falling,
And the silver brooklets brawling,
And the cuckoo-bird is calling
In the spring?

Have you seen a merry bridal in the spring? in the spring?

In an English apple country in the spring?
When the brides and maidens wear
Apple blossoms in their hair:
 Apple blossoms everywhere,
In the spring!

If you have not, then you know not, in the spring, in the spring,
Half the color, beauty, wonder of the spring.
No sight can I remember,
Half so precious, half so tender,
As the apple blossoms render
In the spring!

A MATCH
By A. C. Swinburne

If love were what the rose is,

And I were like the leaf,
Our lives would grow together
In sad or singing weather,
Blown fields or flowerful closes,
Green pastures or gray grief;
If love were what the rose is,
And I were like the leaf.

If you were queen of Pleasure,

And I were king of Pain,
We’d hunt down Love together,
Pluck out his flying-feather,
And teach his feet a measure,
And find his mouth a rein;
If you were queen of Pleasure;
And I were king of Pain.

THE BROOK AND THE WAVE

By Henry Wadsworth Longfellow

The brooklet came from the mountain,

 As sang the bard of old,
Running with feet of silver
Over the sands of gold!

Far away in the briny ocean

There rolled a turbulent wave
Now singing along the sea-beach,
Now howling along the cave.

And the brooklet has found the billow

Though they flowed so far apart,
And has filled with its freshness and sweetness
That turbulent bitter heart!

INDIRECTION
By Richard Realf

Fair are the flowers and the children, but their subtle suggestion is
fairer;
Rare is the roseburst of dawn, but the secret that clasps it is rarer;
Sweet the exultance of song, but the strain that precedes it is
sweeter;
And never was poem yet writ, but the meaning out-mastered the
meter.

Never a daisy that grows, but a mystery guideth the growing;

Never a river that flows, but a majesty scepters the flowing;
Never a Shakespeare that soared, but a stronger than he did enfold
him,
Nor ever a prophet foretells, but a mightier seer hath foretold him.

Back of the canvas that throbs the painter is hinted and hidden;
Into the statue that breathes the soul of the sculptor is bidden;
Under the joy that is felt lie the infinite issues of feeling;
Crowning the glory revealed is the glory that crowns the revealing.
Great are the symbols of being, but that which is symboled is
greater;
Vast the create and beheld, but vaster the inward creator;
Back of the sound broods the silence, back of the gift stands the
giving;
Back of the hand that receives thrill the sensitive nerves of receiving.

Space is as nothing to spirit, the deed is outdone by the doing;

The heart of the wooer is warm, but warmer the heart of the wooing;
And up from the pits where these shiver, and up from the heights
where those shine,
Twin voices and shadows swim starward, and the essence of life is
divine.

—Copyright by Funk & Wagnalls Co., New York, and used by kind
permission.

LIFE AND LOVE

By Richard Realf

There is something to live for and something to love

Wherever we linger, wherever we rove,
There are thousands of sad ones to cheer and sustain
Till hopes that were hidden beam o’er them again.

There is something to live for and something to love,

For the spirit of Man is like garden or grove,
It will yield a sweet fragrance, but still you must toil,
And cherish the blossoms, and culture the soil.

There is something to live for and something to love,

’Tis a truth which the misanthrope ne’er can disprove,
For tho’ thorns and thistles may choke up the flower,
Some beauty will grace the most desolate bower.

Then think on, brother, wherever thou art,

Let the life be for men and the love for the heart,
For know that the pathway which leads us above
Is something to live for and something to love.

—Copyright by Funk & Wagnalls Co., New York, and used by kind
permission.

SONG OF SPRING
By Richard Realf

My heart goes forth to meet the Spring

With the step of a bounding roe,
For it seems like the touch of a seraph’s wing
When the pleasant south winds blow.

O, I love the loveliness that lies

In the smiling heart of May,
The beauty throbbing in violet eyes,
The breath of the fragrant hay.

There’s a great calm joy in the song of birds,

And in the voice of the streams,
In the lowly peace of flocks and herds,
And our own soul’s quiet dreams.

So my heart goes forth to meet the Spring

As a lover to his bride;
And over us both there broods the wing
Of the angel at her side.

—Copyright by Funk & Wagnalls Co., New York, and used by kind
permission.

SONG OF THE SEAMSTRESS

By Richard Realf

It is twelve o’clock by the city’s chime,

 And my task is not yet done;
Through two more weary hours of time
Must my heavy eyes ache on.
I may not suffer my tears to come,
And I dare not stop to feel;
For each idle moment steals a crumb
From my sad to-morrow’s meal.

It is very cold in this cheerless room,

And my limbs are strangely chill;
My pulses beat with a sense of doom,
And my very heart seems still;
But I shall not care for this so much,
If my fingers hold their power,
And the hand of sleep forbears to touch
My eyes for another hour.

I wish I could earn a little more,

And live in another street,
Where I need not tremble to pass the door,
And shudder at all I meet.
’Tis a fearful thing that a friendless girl
Forever alone should dwell
In the midst of scenes enough to hurl
A universe to hell.

God knows that I do not wish to sink

In the pit that yawns around;
But I cannot stand on its very brink,
As I could on purer ground;
I do not think that my strength is gone,
Nor fear for my shortening breath;
But the terrible winter is coming on,
And I must not starve to death.

I wish I had died with sister Rose,

Ere hunger and I were mates;
Ere I felt the grip of the thought that grows
 The hotter the more it waits.
I am sure that He whom they curse to me,
The Father of all our race,
Did not mean the world He made to be
Such a dark and dreary place.

I would not mind if they’d only give

A little less meager pay,
And spare me a moment’s time to grieve,
With a little while to pray.
But until these far-off blessings come,
I may neither weep nor kneel;
For, alas! ’twould cost me a precious crumb
Of my sad to-morrow’s meal.

—Copyright by Funk & Wagnalls Co., New York, and used by kind
permission.

SONG OF THE INDIAN MOTHER

By James Gowdy Clark

Gently dream, my darling child,

Sleeping in the lonely wild;
Would thy dreams might never know
Clouds that darken mine with woe;
Oh! to smile as thou art smiling,
All my hopeless hours beguiling
With the hope that thou mightst see
Blessings that are hid from me.

CHORUS
Lullaby, my gentle boy,
Sleeping in the wilderness,
Dreaming in thy childish joy
Of a mother’s fond caress,—
Lullaby, lullaby.
Sleep, while gleams the council fire,
Kindled by thy hunted sire:
Guarded by thy God above,
Sleep and dream of peace and love:
Dream not of the band that perished
From the sacred soil they cherished,
Nor the ruthless race that roams
O’er our ancient shrines and homes.

Sleep, while autumn glories fly,

’Neath the melancholy sky,
From the trees before the storm,
Chased by winter’s tyrant form:
Oh! ’tis thus our warriors, wasted,
From their altars torn and blasted,
Followed by the storm of death,
Fly before Oppression’s breath.

Sleep, while night hides home and grave,

Rest, while mourn the suff’ring brave,
Mourning as thou, too, wilt mourn,
Through the future, wild and worn;
Bruised in heart, in spirit shaken,
Scourged by man, by God forsaken,
Wandering on in war and strife,
Living still, yet cursing life.

Could thy tender fancy feel

All that manhood will reveal,
Couldst thou dream thy breast would share
All the ills thy fathers bear,
Thou wouldst weep as I am weeping,
Tearful watches wildly keeping,
By the silver-beaming light
Of the long and lonely night.

(Repeat Chorus)
 OLD TIMES
By Gerald Griffin

Old times! old times! the gay old times!

When I was young and free,
And heard the merry Easter chimes
Under the sally tree.
My Sunday palm beside me placed,
My cross upon my hand;
A heart at rest within my breast,
And sunshine on the land!
Old times! old times!

It is not that my fortunes flee,

Nor that my cheek is pale;
I mourn whene’er I think of thee,
My darling native vale!
A wiser head I have, I know,
Than when I loitered there;
But in my wisdom there is woe,
And in my knowledge care.
Old times! old times!

I’ve lived to know my share of joy,

To feel my share of pain;
To learn that friendship’s self can cloy,
To love and love in vain;
To feel a pang and wear a smile,
To tire of other climes;
To love my own unhappy Isle,
And sing the gay old times!
Old times! old times!

And sure the land is nothing changed;

The birds are singing still,
The flowers are springing where we ranged,
There’s sunshine on the hill.
The sally waving o’er my head
Still sweetly shades my frame;
But oh! those happy days are fled,
And I am not the same.
Old times! old times!

Oh, come again, ye merry times!

Sweet, sunny, fresh and calm;
And let me hear those Easter chimes,
And wear my Sunday palm.
If I could cry away mine eyes,
My tears would flow in vain;
If I could waste my heart in sighs,
They’ll never come again!
Old times! old times!

TWILIGHT FANCIES
By Eliza A. Pittsinger

Softly flit the fairy fancies

Through the sunlight of my brain,
Weaving webs of weird romances
In a laughing, joyous strain—
Gently creeping,
Gaily leaping,
Twilight revels strangely keeping
In my brain.

Ere the evening lamps are lighted,

While my soul is wrapt in thought,
Wait they not to be invited,
Quite unwelcome and unsought—
Never sitting,
Ever flitting,
All the earnestness outwitting
Of my thought.
Thus to have my being haunted
By these fairies, all astray,
By these elfin-sprites enchanted,
Is a spell upon my way,
That shall borrow
For the morrow,
All the pleasure and the sorrow
Of to-day.

In my hours of quiet musing,

By these phantoms thus caressed,
I have lost the right of choosing
As I ought, my favored guest—
Uninvited,
Often slighted,
Come they when the lamps are lighted
For a guest.

Thus they come, the fairy fancies,

Laughing, flitting through my brain,
Weaving webs of weird romances,
In a wayward, joyous strain—
Gaily creeping,
Madly leaping,
Even now their revels keeping
In my brain.

THE SONG OF THE FLUME[16]

By Anna M. Fitch

Awake, awake! for my track is red,

With the glow of the coming day;
And with tinkling tread, from my dusky bed,
I haste o’er hill away,
Up from the valley, up from the plain,
Up from the river’s side;
For I come with a gush, and a torrent’s rush,
And there’s wealth in my swelling tide.

I am fed by the melting rills that start

Where the sparkling snow-peaks gleam,
My voice is free, and with fiercest glee
I leap in the sun’s broad beam;
Tho’ torn from the channels deep and old,
I have worn through the craggy hill,
Yet I flow in pride, as my waters glide,
And there’s mirth in my music still.

I sought the shore of the sounding sea,

From the far Sierra’s height,
With a starry breast, and a snow-capped crest
I foamed in a path of light;
But they bore me thence in a winding way,
They’ve fettered me like a slave,
And as scarfs of old were exchanged for gold,
So they barter my soil-stained wave.

Thro’ the deep tunnel, down the dark shaft,

I search for the shining ore;
Hoist it away to the light of day,
Which it never has seen before.
Spade and shovel, mattock and pick,
Ply them with eager haste;
For my golden shower is sold by the hour,
And the drops are too dear to waste.

Lift me aloft to the mountain’s brow,

Fathom the deep “blue vein,”
And I’ll sift the soil for the shining spoil,
As I sink to the valley again.
The swell of my swarthy breast shall bear
Pebble and rock away,
Though they brave my strength, they shall yield at length,
But the glittering gold shall stay.
Mine is no stern and warrior march,
No stormy trump and drum;
No banners gleam in my darkened stream,
As with conquering step I come;
But I touch the tributary earth
Till it owns a monarch’s sway,
And with eager hand, from a conquered land,
I bear its wealth away.

Awake, awake! there are loving hearts

In the lands you’ve left afar;
There are tearful eyes in the homes you prize
As they gaze on the western star;
Then up from the valley, up from the hill,
Up from the river’s side;
For I come with a gush, and a torrent’s rush,
And there’s wealth in my swelling tide.

THE WEST
By Annie Elizabeth Cheney

Wings that are glancing, wings of my soul,

That speeding like arrows fly to their goal;
Wings that have cut the keen ethers above,
O carry me on to the West of my love!

The West it is magic, perspective and fire,

Its peaks are like daggers thrust up by desire;
It is Tyre, it is Sidon and Ophir in one,
This land by the waters, this land of the sun.

—From “Dreams of Hellas.”

THE MOON-CRADLE
By Kate Wisner M’Cluskey
The little, the yellow moon-cradle
Is swaying, is swinging slow;
And the tiny white star-tapers burning
Have flickered their lights down low;
The night has the cloud-curtains ready,
She is holding them draped on her breast,
For the dear little, queer little babe in the moon
Will have sunk to rest in the west.
Hush, baby, hush!
Mother’s heart aches for the joy that she takes
In holding you close to her breast!

Perhaps in the yellow moon-cradle

A little cold baby may be;
And the tiny white star-tapers burning
May be sad for some mother to see;—
O night-angel! drop the cloud-curtain
While the gleaming bed’s caught in that tree,
For not even to the rest in the beautiful west
Will I let my babe go from me!
Sleep, sleep, my sweet!
Are you warm, little feet?
Close to my heart you will be!

GREEN THINGS GROWING

By Dinah Maria Mulock

Oh, the green things growing, the green things growing,

The faint sweet smell of the green things growing!
I should like to live, whether I smile or grieve,
Just to watch the happy life of my green things growing.

Oh, the fluttering and the pattering of those green things growing.
How they talk each to each, when none of us are knowing
In the wonderful white of the weird moonlight
Or the dim, dreary dawn when the cocks are crowing.