Ygtjki

Is it inappropriate for a student to attend their mentor's dissertation defense?

Am I breaking OOP practice with this architecture?

Ambiguity in the definition of entropy

How much of data wrangling is a data scientist's job?

Could the museum Saturn V's be refitted for one more flight?

Gatling : Performance testing tool

Watching something be piped to a file live with tail

Dreadful Dastardly Diseases, or Always Atrocious Ailments

How do I know where to place holes on an instrument?

Is it acceptable for a professor to tell male students to not think that they are smarter than female students?

Assassin's bullet with mercury

Is "remove commented out code" correct English?

Bullying boss launched a smear campaign and made me unemployable

Im going to France and my passport expires June 19th

Detention in 1997

Intersection Puzzle

How did the Super Star Destroyer Executor get destroyed exactly?

Personal Teleportation: From Rags to Riches

What is a romance in Latin?

How do I gain back my faith in my PhD degree?

Venezuelan girlfriend wants to travel the USA to be with me. What is the process?

How seriously should I take size and weight limits of hand luggage?

Zip/Tar file compressed to larger size?

Why can't we play rap on piano?

Variance of Monte Carlo integration with importance sampling

Monte Carlo Integration for non-square integrable functionsMonte Carlo integration aim for maximum varianceVariance reduction technique in Monte Carlo integrationMonte Carlo integration and varianceMonte Carlo Integration on the Real LineUse Importance Sampling and Monte carlo for estimating a summationSampling / Importance Resampling Poisson WeightsImportance SamplingFind the value of an integral using Monte-Carlo methodOptimal proposal for self-normalized importance sampling

3

3

$begingroup$

I am following these lecture slides on Monte Carlo integration with importance sampling. I am just implementing a very simple example: $int_0^1 e^xdx$. For the importance sampling version, I rewrite $int_0^1 e^xdx = int_0^1 e^x/p(x)cdot p(x)dx$ where $p(x) = 2.5x^1.5$. Then

$$hatI = frac1Nsum_j=1^N fracf(x_j)p(x_j),$$

where $x_j$ are sampled from $p(x_j)$ (I use an inverse transform method here). For the variance, I have $sigma_I^2 = hatsigma_I^2/N$ and

$$hatsigma_I^2 = frac1N sum_j=1^N fracf(x_j)^2g(x_j)^2 - hatI^2.$$

I know I should expected the variance to decrease with importance sampling, but a plot of the variance with $N$ shows that not much happens. Can anyone explain to me what I'm doing incorrectly? I'm not sure how the they are able to achieve such a drastic decrease in variance in the lecture slides.

monte-carlo integral importance-sampling

share|cite|improve this question

asked 2 days ago

user1799323user1799323

1234

$endgroup$

add a comment | 

3

3

$begingroup$

I am following these lecture slides on Monte Carlo integration with importance sampling. I am just implementing a very simple example: $int_0^1 e^xdx$. For the importance sampling version, I rewrite $int_0^1 e^xdx = int_0^1 e^x/p(x)cdot p(x)dx$ where $p(x) = 2.5x^1.5$. Then

$$hatI = frac1Nsum_j=1^N fracf(x_j)p(x_j),$$

where $x_j$ are sampled from $p(x_j)$ (I use an inverse transform method here). For the variance, I have $sigma_I^2 = hatsigma_I^2/N$ and

$$hatsigma_I^2 = frac1N sum_j=1^N fracf(x_j)^2g(x_j)^2 - hatI^2.$$

I know I should expected the variance to decrease with importance sampling, but a plot of the variance with $N$ shows that not much happens. Can anyone explain to me what I'm doing incorrectly? I'm not sure how the they are able to achieve such a drastic decrease in variance in the lecture slides.

monte-carlo integral importance-sampling

share|cite|improve this question

asked 2 days ago

user1799323user1799323

1234

$endgroup$

add a comment | 

$begingroup$

I am following these lecture slides on Monte Carlo integration with importance sampling. I am just implementing a very simple example: $int_0^1 e^xdx$. For the importance sampling version, I rewrite $int_0^1 e^xdx = int_0^1 e^x/p(x)cdot p(x)dx$ where $p(x) = 2.5x^1.5$. Then

$$hatI = frac1Nsum_j=1^N fracf(x_j)p(x_j),$$

where $x_j$ are sampled from $p(x_j)$ (I use an inverse transform method here). For the variance, I have $sigma_I^2 = hatsigma_I^2/N$ and

$$hatsigma_I^2 = frac1N sum_j=1^N fracf(x_j)^2g(x_j)^2 - hatI^2.$$

I know I should expected the variance to decrease with importance sampling, but a plot of the variance with $N$ shows that not much happens. Can anyone explain to me what I'm doing incorrectly? I'm not sure how the they are able to achieve such a drastic decrease in variance in the lecture slides.

monte-carlo integral importance-sampling

share|cite|improve this question

asked 2 days ago

user1799323user1799323

1234

$endgroup$

share|cite|improve this question

asked 2 days ago

user1799323user1799323

1234

share|cite|improve this question

asked 2 days ago

user1799323user1799323

1234

asked 2 days ago

user1799323user1799323

1234

asked 2 days ago

user1799323user1799323

1234

1 Answer
1

active

oldest

votes

3

$begingroup$

This is a good illustration of the dangers of importance sampling: while
$$int_0^1 frace^xp(x), p(x)textd x = int_0^1 e^x textd x = I$$
shows that $hatI_N$ is an unbiased estimator of $I$, this estimator does not have a finite variance since
$$int_0^1 left(frace^xp(x)right)^2, p(x)textd x = int_0^1 frace^2x2.5 x^1.5 textd x = infty$$
since the integral diverges in $x=0$. For instance,

> x=runif(1e7)^1/2.5
> range(exp(x)/x^1.5)
[1] 2.718282 83403.685972

shows that the weights can widely differ. I am not surprised at the figures reported in the above slides since

 > mean(exp(x)/x^1.5)/2.5
 [1] 1.717576
 > var(exp(x)/x^1.5)/(2.5)^2/1e7
 [1] 2.070953e-06

but the empirical variance is rarely able to spot infinite variance importance sampling. (The graph shows that both the standard Monte Carlo estimate and the importance sampling version see the empirical standard deviation is decreasing as $N^-1/2$.)

share|cite|improve this answer

edited 2 days ago

answered 2 days ago

Xi'anXi'an

59.1k897365

$endgroup$

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "65"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f400628%2fvariance-of-monte-carlo-integration-with-importance-sampling%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

3

$begingroup$

This is a good illustration of the dangers of importance sampling: while
$$int_0^1 frace^xp(x), p(x)textd x = int_0^1 e^x textd x = I$$
shows that $hatI_N$ is an unbiased estimator of $I$, this estimator does not have a finite variance since
$$int_0^1 left(frace^xp(x)right)^2, p(x)textd x = int_0^1 frace^2x2.5 x^1.5 textd x = infty$$
since the integral diverges in $x=0$. For instance,

> x=runif(1e7)^1/2.5
> range(exp(x)/x^1.5)
[1] 2.718282 83403.685972

shows that the weights can widely differ. I am not surprised at the figures reported in the above slides since

 > mean(exp(x)/x^1.5)/2.5
 [1] 1.717576
 > var(exp(x)/x^1.5)/(2.5)^2/1e7
 [1] 2.070953e-06

but the empirical variance is rarely able to spot infinite variance importance sampling. (The graph shows that both the standard Monte Carlo estimate and the importance sampling version see the empirical standard deviation is decreasing as $N^-1/2$.)

share|cite|improve this answer

edited 2 days ago

answered 2 days ago

Xi'anXi'an

59.1k897365

$endgroup$

add a comment | 

3

$begingroup$

This is a good illustration of the dangers of importance sampling: while
$$int_0^1 frace^xp(x), p(x)textd x = int_0^1 e^x textd x = I$$
shows that $hatI_N$ is an unbiased estimator of $I$, this estimator does not have a finite variance since
$$int_0^1 left(frace^xp(x)right)^2, p(x)textd x = int_0^1 frace^2x2.5 x^1.5 textd x = infty$$
since the integral diverges in $x=0$. For instance,

> x=runif(1e7)^1/2.5
> range(exp(x)/x^1.5)
[1] 2.718282 83403.685972

shows that the weights can widely differ. I am not surprised at the figures reported in the above slides since

 > mean(exp(x)/x^1.5)/2.5
 [1] 1.717576
 > var(exp(x)/x^1.5)/(2.5)^2/1e7
 [1] 2.070953e-06

but the empirical variance is rarely able to spot infinite variance importance sampling. (The graph shows that both the standard Monte Carlo estimate and the importance sampling version see the empirical standard deviation is decreasing as $N^-1/2$.)

share|cite|improve this answer

edited 2 days ago

answered 2 days ago

Xi'anXi'an

59.1k897365

$endgroup$

add a comment | 

$begingroup$

This is a good illustration of the dangers of importance sampling: while
$$int_0^1 frace^xp(x), p(x)textd x = int_0^1 e^x textd x = I$$
shows that $hatI_N$ is an unbiased estimator of $I$, this estimator does not have a finite variance since
$$int_0^1 left(frace^xp(x)right)^2, p(x)textd x = int_0^1 frace^2x2.5 x^1.5 textd x = infty$$
since the integral diverges in $x=0$. For instance,

> x=runif(1e7)^1/2.5
> range(exp(x)/x^1.5)
[1] 2.718282 83403.685972

shows that the weights can widely differ. I am not surprised at the figures reported in the above slides since

 > mean(exp(x)/x^1.5)/2.5
 [1] 1.717576
 > var(exp(x)/x^1.5)/(2.5)^2/1e7
 [1] 2.070953e-06

but the empirical variance is rarely able to spot infinite variance importance sampling. (The graph shows that both the standard Monte Carlo estimate and the importance sampling version see the empirical standard deviation is decreasing as $N^-1/2$.)

share|cite|improve this answer

edited 2 days ago

answered 2 days ago

Xi'anXi'an

59.1k897365

$endgroup$

> x=runif(1e7)^1/2.5
> range(exp(x)/x^1.5)
[1] 2.718282 83403.685972

 > mean(exp(x)/x^1.5)/2.5
 [1] 1.717576
 > var(exp(x)/x^1.5)/(2.5)^2/1e7
 [1] 2.070953e-06

share|cite|improve this answer

edited 2 days ago

answered 2 days ago

Xi'anXi'an

59.1k897365

answered 2 days ago

Xi'anXi'an

59.1k897365

answered 2 days ago

Xi'anXi'an

59.1k897365