Etydhv

My parents are Afghan

How is it believable that Euron could so easily pull off this ambush?

How can it be that ssh somename works, while nslookup somename does not?

Is there an application which does HTTP PUT?

Two (probably) equal real numbers which are not proved to be equal?

Magical Modulo Squares

How to start your Starctaft II games vs AI immediatly?

Crime rates in a post-scarcity economy

I want to write a blog post building upon someone else's paper, how can I properly cite/credit them?

Efficient manipulation of Associations passed to functions, how to?

Is there an idiom that means "revealing a secret unintentionally"?

Using mean length and mean weight to calculate mean BMI?

Should one save up to purchase a house/condo or maximize their 401k first?

Is it safe to keep the GPU on 100% utilization for a very long time?

Why is the episode called "The Last of the Starks"?

How do I minimise waste on a flight?

Do these creatures from the Tomb of Annihilation campaign speak Common?

Where do 5 or more U.S. counties meet in a single point?

Gift for mentor after his thesis defense?

When an electron around an atom drops to a lower state, is 100% of the energy converted to a photon?

Can the Telekinesis spell be used on yourself for the following?

Whose birthyears are canonically established in the MCU?

How do I give a darkroom course without negatives from the attendees?

get unsigned long long addition carry

Efficient manipulation of Associations passed to functions, how to?

When must I use the Return function?How can I implement object oriented programming in Mathematica?Pascal records and Mathematica programmingRepeatedly randomly redrawing part of a random sample that matches a certain criterionQuestion about designing a particular data structureWhat are the most common pitfalls awaiting new users?How to speed up minimization of functionDouble For loop problemThe curious case of missing random-walking particles in the boxHow to fix this issue with storing an expression as a String and returning it with ToExpressionListDensityPlot behaviour and gnuplot's splot analogueMulti-dimensional PositionIndexPascal records and Mathematica programming

1

1

$begingroup$

I have some data wrapped into a MyData[data_Association] "structure".

My Association contains some big arrays and I do not want to copy them each time I modify it.

By example, if I want to add a field, does the following code

foo[MyData[data_Association]] :=
 Block[datacpy,
 datacpy = data;
 datacpy["extraField"] = 1;
 Return[MyData[datacpy]];
 ];

data=<|"bigArray"->|>; (* not empty in real situation! *)

foo[MyData[data]]

MyData[<|"bigArray" -> , "extraField" ->
1|>]

copy the "bigArray" field even if it is not modified? (or does it use some binding/reference mechanism).

performance-tuning programming

share|improve this question

edited 1 hour ago

Picaud Vincent

asked 2 hours ago

Picaud VincentPicaud Vincent

1,354518

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Maybe store the associations not in an array but in a dictionary? (like dict[key]=key->value) and store dictionary globally?
  $endgroup$
  – Kagaratsch
  2 hours ago
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  I find the "wrapping" trick SomeSymbol[data...] very useful and I wanted to keep using this simple approach. But thanks for the suggestion.
  $endgroup$
  – Picaud Vincent
  2 hours ago
  

  
  
  
  

add a comment | 

1

1

$begingroup$

I have some data wrapped into a MyData[data_Association] "structure".

My Association contains some big arrays and I do not want to copy them each time I modify it.

By example, if I want to add a field, does the following code

foo[MyData[data_Association]] :=
 Block[datacpy,
 datacpy = data;
 datacpy["extraField"] = 1;
 Return[MyData[datacpy]];
 ];

data=<|"bigArray"->|>; (* not empty in real situation! *)

foo[MyData[data]]

MyData[<|"bigArray" -> , "extraField" ->
1|>]

copy the "bigArray" field even if it is not modified? (or does it use some binding/reference mechanism).

performance-tuning programming

share|improve this question

edited 1 hour ago

Picaud Vincent

asked 2 hours ago

Picaud VincentPicaud Vincent

1,354518

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Maybe store the associations not in an array but in a dictionary? (like dict[key]=key->value) and store dictionary globally?
  $endgroup$
  – Kagaratsch
  2 hours ago
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  I find the "wrapping" trick SomeSymbol[data...] very useful and I wanted to keep using this simple approach. But thanks for the suggestion.
  $endgroup$
  – Picaud Vincent
  2 hours ago
  

  
  
  
  

add a comment | 

$begingroup$

I have some data wrapped into a MyData[data_Association] "structure".

My Association contains some big arrays and I do not want to copy them each time I modify it.

By example, if I want to add a field, does the following code

foo[MyData[data_Association]] :=
 Block[datacpy,
 datacpy = data;
 datacpy["extraField"] = 1;
 Return[MyData[datacpy]];
 ];

data=<|"bigArray"->|>; (* not empty in real situation! *)

foo[MyData[data]]

MyData[<|"bigArray" -> , "extraField" ->
1|>]

copy the "bigArray" field even if it is not modified? (or does it use some binding/reference mechanism).

performance-tuning programming

share|improve this question

edited 1 hour ago

Picaud Vincent

asked 2 hours ago

Picaud VincentPicaud Vincent

1,354518

$endgroup$

foo[MyData[data_Association]] :=
 Block[datacpy,
 datacpy = data;
 datacpy["extraField"] = 1;
 Return[MyData[datacpy]];
 ];

data=<|"bigArray"->|>; (* not empty in real situation! *)

foo[MyData[data]]

share|improve this question

edited 1 hour ago

Picaud Vincent

asked 2 hours ago

Picaud VincentPicaud Vincent

1,354518

share|improve this question

edited 1 hour ago

Picaud Vincent

asked 2 hours ago

Picaud VincentPicaud Vincent

1,354518

asked 2 hours ago

Picaud VincentPicaud Vincent

1,354518

asked 2 hours ago

Picaud VincentPicaud Vincent

1,354518

2 Answers
2

active

oldest

votes

2

$begingroup$

I think you're asking whether the foo operation will store two independent copies of the big array. That is, given:

MemoryInUse[]
ds = MyData @ Association["BigArray"->RandomReal[1,10^8]];
MemoryInUse[]

42312736

842355744

Will your foo operation basically double the memory in use? Here is your foo function (rewritten to eliminate the superfluous Return that should never be used at the end of a Module or Block):

foo[MyData[data_Association]] := Block[datacpy = data,
 datacpy["extraField"] = 1;
 MyData[datacpy]
]

Then:

MemoryInUse[]
foo[ds];
MemoryInUse[]

845119768

845122344

I think the above shows that the answer is no.

share|improve this answer

answered 2 hours ago

Carl WollCarl Woll

77.5k3102204

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  I feel really uncomfortable not knowing about Return[] usage -> I just found this mathematica.stackexchange.com/questions/58059/… Hopefully for my mood the post starts with "Return is surprisingly under-documented"
  $endgroup$
  – Picaud Vincent
  1 hour ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  Return is almost useless. It isn't really under-documented, but if you expect it to be like "return" in other languages, you'll find nothing to support such usage.
  $endgroup$
  – John Doty
  1 hour ago
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @JohnDoty thanks, this is indeed the trap I fell into.
  $endgroup$
  – Picaud Vincent
  1 hour ago
  

  
  
  
  

add a comment | 

1

$begingroup$

It all depends on how you want to do the modification. In general, Mathematica is actually very efficient about handling pass-by-value calls. If you look at memory consumption "bigArray" will not be copied:

struct[] :=
 struct[<||>];
struct[a_]@"Add"[field_ , val_] :=
 struct[Append[a, field -> val]];
struct[a_]@"Modify"[field_ , fn_] :=
 struct[ReplacePart[a, field -> fn@a[field]]];

myStruct = struct[];

Block[arr = RandomReal[, 500, 500],
 (* done to prevent the memory getting stuck to Out *)
 myStruct = myStruct@"Add"["bigArray", arr];
 arr // ByteCount, memPrev = MemoryInUse[]
 ]

2000152, 974574520

Block[,
 (* done to prevent the memory getting stuck to Out *)
 myStruct = myStruct@"Add"["empty", None];
 MemoryInUse[] - memPrev
 ]

3472

And in fact "bigArray" is really stored as an object with an internal ID as can be seen in the many low-level functions that depend on explicit expression identity.

Of course, you could do this with a direct mutable OOP approach, too, but I won't get into that here.

share|improve this answer

answered 2 hours ago

b3m2a1b3m2a1

29.4k360172

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  By mutable OOP approach you mean something like using HoldFirst attribute etc... ?
  $endgroup$
  – Picaud Vincent
  2 hours ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  @PicaudVincent basically. And using a Symbol to store state. See e.g. this: mathematica.stackexchange.com/a/165486/38205
  $endgroup$
  – b3m2a1
  2 hours ago
  
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  If you like OOP-style programming I also have the interface package that I used for this: mathematica.stackexchange.com/a/195065/38205
  $endgroup$
  – b3m2a1
  2 hours ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thanks for the clarification.
  $endgroup$
  – Picaud Vincent
  2 hours ago
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "387"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmathematica.stackexchange.com%2fquestions%2f197884%2fefficient-manipulation-of-associations-passed-to-functions-how-to%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

2

$begingroup$

I think you're asking whether the foo operation will store two independent copies of the big array. That is, given:

MemoryInUse[]
ds = MyData @ Association["BigArray"->RandomReal[1,10^8]];
MemoryInUse[]

42312736

842355744

Will your foo operation basically double the memory in use? Here is your foo function (rewritten to eliminate the superfluous Return that should never be used at the end of a Module or Block):

foo[MyData[data_Association]] := Block[datacpy = data,
 datacpy["extraField"] = 1;
 MyData[datacpy]
]

Then:

MemoryInUse[]
foo[ds];
MemoryInUse[]

845119768

845122344

I think the above shows that the answer is no.

share|improve this answer

answered 2 hours ago

Carl WollCarl Woll

77.5k3102204

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  I feel really uncomfortable not knowing about Return[] usage -> I just found this mathematica.stackexchange.com/questions/58059/… Hopefully for my mood the post starts with "Return is surprisingly under-documented"
  $endgroup$
  – Picaud Vincent
  1 hour ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  Return is almost useless. It isn't really under-documented, but if you expect it to be like "return" in other languages, you'll find nothing to support such usage.
  $endgroup$
  – John Doty
  1 hour ago
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @JohnDoty thanks, this is indeed the trap I fell into.
  $endgroup$
  – Picaud Vincent
  1 hour ago
  

  
  
  
  

add a comment | 

2

$begingroup$

I think you're asking whether the foo operation will store two independent copies of the big array. That is, given:

MemoryInUse[]
ds = MyData @ Association["BigArray"->RandomReal[1,10^8]];
MemoryInUse[]

42312736

842355744

Will your foo operation basically double the memory in use? Here is your foo function (rewritten to eliminate the superfluous Return that should never be used at the end of a Module or Block):

foo[MyData[data_Association]] := Block[datacpy = data,
 datacpy["extraField"] = 1;
 MyData[datacpy]
]

Then:

MemoryInUse[]
foo[ds];
MemoryInUse[]

845119768

845122344

I think the above shows that the answer is no.

share|improve this answer

answered 2 hours ago

Carl WollCarl Woll

77.5k3102204

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  I feel really uncomfortable not knowing about Return[] usage -> I just found this mathematica.stackexchange.com/questions/58059/… Hopefully for my mood the post starts with "Return is surprisingly under-documented"
  $endgroup$
  – Picaud Vincent
  1 hour ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  Return is almost useless. It isn't really under-documented, but if you expect it to be like "return" in other languages, you'll find nothing to support such usage.
  $endgroup$
  – John Doty
  1 hour ago
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @JohnDoty thanks, this is indeed the trap I fell into.
  $endgroup$
  – Picaud Vincent
  1 hour ago
  

  
  
  
  

add a comment | 

$begingroup$

I think you're asking whether the foo operation will store two independent copies of the big array. That is, given:

MemoryInUse[]
ds = MyData @ Association["BigArray"->RandomReal[1,10^8]];
MemoryInUse[]

42312736

842355744

Will your foo operation basically double the memory in use? Here is your foo function (rewritten to eliminate the superfluous Return that should never be used at the end of a Module or Block):

foo[MyData[data_Association]] := Block[datacpy = data,
 datacpy["extraField"] = 1;
 MyData[datacpy]
]

Then:

MemoryInUse[]
foo[ds];
MemoryInUse[]

845119768

845122344

I think the above shows that the answer is no.

share|improve this answer

answered 2 hours ago

Carl WollCarl Woll

77.5k3102204

$endgroup$

MemoryInUse[]
ds = MyData @ Association["BigArray"->RandomReal[1,10^8]];
MemoryInUse[]

foo[MyData[data_Association]] := Block[datacpy = data,
 datacpy["extraField"] = 1;
 MyData[datacpy]
]

MemoryInUse[]
foo[ds];
MemoryInUse[]

share|improve this answer

answered 2 hours ago

Carl WollCarl Woll

77.5k3102204

answered 2 hours ago

Carl WollCarl Woll

77.5k3102204

answered 2 hours ago

Carl WollCarl Woll

77.5k3102204

1

$begingroup$

It all depends on how you want to do the modification. In general, Mathematica is actually very efficient about handling pass-by-value calls. If you look at memory consumption "bigArray" will not be copied:

struct[] :=
 struct[<||>];
struct[a_]@"Add"[field_ , val_] :=
 struct[Append[a, field -> val]];
struct[a_]@"Modify"[field_ , fn_] :=
 struct[ReplacePart[a, field -> fn@a[field]]];

myStruct = struct[];

Block[arr = RandomReal[, 500, 500],
 (* done to prevent the memory getting stuck to Out *)
 myStruct = myStruct@"Add"["bigArray", arr];
 arr // ByteCount, memPrev = MemoryInUse[]
 ]

2000152, 974574520

Block[,
 (* done to prevent the memory getting stuck to Out *)
 myStruct = myStruct@"Add"["empty", None];
 MemoryInUse[] - memPrev
 ]

3472

And in fact "bigArray" is really stored as an object with an internal ID as can be seen in the many low-level functions that depend on explicit expression identity.

Of course, you could do this with a direct mutable OOP approach, too, but I won't get into that here.

share|improve this answer

answered 2 hours ago

b3m2a1b3m2a1

29.4k360172

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  By mutable OOP approach you mean something like using HoldFirst attribute etc... ?
  $endgroup$
  – Picaud Vincent
  2 hours ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  @PicaudVincent basically. And using a Symbol to store state. See e.g. this: mathematica.stackexchange.com/a/165486/38205
  $endgroup$
  – b3m2a1
  2 hours ago
  
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  If you like OOP-style programming I also have the interface package that I used for this: mathematica.stackexchange.com/a/195065/38205
  $endgroup$
  – b3m2a1
  2 hours ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thanks for the clarification.
  $endgroup$
  – Picaud Vincent
  2 hours ago
  

  
  
  
  

add a comment | 

1

$begingroup$

It all depends on how you want to do the modification. In general, Mathematica is actually very efficient about handling pass-by-value calls. If you look at memory consumption "bigArray" will not be copied:

struct[] :=
 struct[<||>];
struct[a_]@"Add"[field_ , val_] :=
 struct[Append[a, field -> val]];
struct[a_]@"Modify"[field_ , fn_] :=
 struct[ReplacePart[a, field -> fn@a[field]]];

myStruct = struct[];

Block[arr = RandomReal[, 500, 500],
 (* done to prevent the memory getting stuck to Out *)
 myStruct = myStruct@"Add"["bigArray", arr];
 arr // ByteCount, memPrev = MemoryInUse[]
 ]

2000152, 974574520

Block[,
 (* done to prevent the memory getting stuck to Out *)
 myStruct = myStruct@"Add"["empty", None];
 MemoryInUse[] - memPrev
 ]

3472

And in fact "bigArray" is really stored as an object with an internal ID as can be seen in the many low-level functions that depend on explicit expression identity.

Of course, you could do this with a direct mutable OOP approach, too, but I won't get into that here.

share|improve this answer

answered 2 hours ago

b3m2a1b3m2a1

29.4k360172

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  By mutable OOP approach you mean something like using HoldFirst attribute etc... ?
  $endgroup$
  – Picaud Vincent
  2 hours ago
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  @PicaudVincent basically. And using a Symbol to store state. See e.g. this: mathematica.stackexchange.com/a/165486/38205
  $endgroup$
  – b3m2a1
  2 hours ago
  
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  If you like OOP-style programming I also have the interface package that I used for this: mathematica.stackexchange.com/a/195065/38205
  $endgroup$
  – b3m2a1
  2 hours ago
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thanks for the clarification.
  $endgroup$
  – Picaud Vincent
  2 hours ago
  

  
  
  
  

add a comment | 

$begingroup$

It all depends on how you want to do the modification. In general, Mathematica is actually very efficient about handling pass-by-value calls. If you look at memory consumption "bigArray" will not be copied:

struct[] :=
 struct[<||>];
struct[a_]@"Add"[field_ , val_] :=
 struct[Append[a, field -> val]];
struct[a_]@"Modify"[field_ , fn_] :=
 struct[ReplacePart[a, field -> fn@a[field]]];

myStruct = struct[];

Block[arr = RandomReal[, 500, 500],
 (* done to prevent the memory getting stuck to Out *)
 myStruct = myStruct@"Add"["bigArray", arr];
 arr // ByteCount, memPrev = MemoryInUse[]
 ]

2000152, 974574520

Block[,
 (* done to prevent the memory getting stuck to Out *)
 myStruct = myStruct@"Add"["empty", None];
 MemoryInUse[] - memPrev
 ]

3472

And in fact "bigArray" is really stored as an object with an internal ID as can be seen in the many low-level functions that depend on explicit expression identity.

Of course, you could do this with a direct mutable OOP approach, too, but I won't get into that here.

share|improve this answer

answered 2 hours ago

b3m2a1b3m2a1

29.4k360172

$endgroup$

struct[] :=
 struct[<||>];
struct[a_]@"Add"[field_ , val_] :=
 struct[Append[a, field -> val]];
struct[a_]@"Modify"[field_ , fn_] :=
 struct[ReplacePart[a, field -> fn@a[field]]];

myStruct = struct[];

Block[arr = RandomReal[, 500, 500],
 (* done to prevent the memory getting stuck to Out *)
 myStruct = myStruct@"Add"["bigArray", arr];
 arr // ByteCount, memPrev = MemoryInUse[]
 ]

2000152, 974574520

Block[,
 (* done to prevent the memory getting stuck to Out *)
 myStruct = myStruct@"Add"["empty", None];
 MemoryInUse[] - memPrev
 ]

3472

share|improve this answer

answered 2 hours ago

b3m2a1b3m2a1

29.4k360172

answered 2 hours ago

b3m2a1b3m2a1

29.4k360172

answered 2 hours ago

b3m2a1b3m2a1

29.4k360172