Bdtjtk

1. The accepted solution won't work directly for python3 - it will need an from functools import reduce.


2. Also it seems more pythonic to use a for loop.
   See the quote from What’s New In Python 3.0.
   
   

   
   

   Removed reduce(). Use functools.reduce() if you really need it; however, 99 percent of the time an explicit for loop is more readable.

   
   

   
   


3. Next, the accepted solution doesn't set non-existing nested keys (it returns a KeyError) - see @eafit's answer for a solution

So why not use the suggested method from kolergy's question for getting a value:

def getFromDict(dataDict, mapList):    

    for k in mapList: dataDict = dataDict[k]

    return dataDict

And the code from @eafit's answer for setting a value:

def nested_set(dic, keys, value):

    for key in keys[:-1]:

        dic = dic.setdefault(key, {})

    dic[keys[-1]] = value

Both work straight in python 2 and 3

Using reduce is clever, but the OP's set method may have issues if the parent keys do not pre-exist in the nested dictionary. Since this is the first SO post I saw for this subject in my google search, I would like to make it slightly better.

The set method in ( Setting a value in a nested python dictionary given a list of indices and value ) seems more robust to missing parental keys. To copy it over:

def nested_set(dic, keys, value):

    for key in keys[:-1]:

        dic = dic.setdefault(key, {})

    dic[keys[-1]] = value

Also, it can be convenient to have a method that traverses the key tree and get all the absolute key paths, for which I have created:

def keysInDict(dataDict, parent=):

    if not isinstance(dataDict, dict):

        return [tuple(parent)]

    else:

        return reduce(list.__add__, 

            [keysInDict(v,parent+[k]) for k,v in dataDict.items()], )

One use of it is to convert the nested tree to a pandas DataFrame, using the following code (assuming that all leafs in the nested dictionary have the same depth).

def dict_to_df(dataDict):

    ret = 

    for k in keysInDict(dataDict):

        v = np.array( getFromDict(dataDict, k), )

        v = pd.DataFrame(v)

        v.columns = pd.MultiIndex.from_product(list(k) + [v.columns])

        ret.append(v)

    return reduce(pd.DataFrame.join, ret)

This library may be helpful: https://github.com/akesterson/dpath-python

A python library for accessing and searching dictionaries via
/slashed/paths ala xpath

Basically it lets you glob over a dictionary as if it were a
filesystem.

Instead of taking a performance hit each time you want to look up a value, how about you flatten the dictionary once then simply look up the key like b:v:y

def flatten(mydict):

  new_dict = {}

  for key,value in mydict.items():

    if type(value) == dict:

      _dict = {':'.join([key, _key]):_value for _key, _value in flatten(value).items()}

      new_dict.update(_dict)

    else:

      new_dict[key]=value

  return new_dict



dataDict = {

"a":{

    "r": 1,

    "s": 2,

    "t": 3

    },

"b":{

    "u": 1,

    "v": {

        "x": 1,

        "y": 2,

        "z": 3

    },

    "w": 3

    }

}    



flat_dict = flatten(dataDict)

print flat_dict

{'b:w': 3, 'b:u': 1, 'b:v:y': 2, 'b:v:x': 1, 'b:v:z': 3, 'a:r': 1, 'a:s': 2, 'a:t': 3}

This way you can simply look up items using flat_dict['b:v:y'] which will give you 1.

And instead of traversing the dictionary on each lookup, you may be able to speed this up by flattening the dictionary and saving the output so that a lookup from cold start would mean loading up the flattened dictionary and simply performing a key/value lookup with no traversal.

How about using recursive functions?

To get a value:

def getFromDict(dataDict, maplist):

    first, rest = maplist[0], maplist[1:]



    if rest: 

        # if `rest` is not empty, run the function recursively

        return getFromDict(dataDict[first], rest)

    else:

        return dataDict[first]

And to set a value:

def setInDict(dataDict, maplist, value):

    first, rest = maplist[0], maplist[1:]



    if rest:

        try:

            if not isinstance(dataDict[first], dict):

                # if the key is not a dict, then make it a dict

                dataDict[first] = {}

        except KeyError:

            # if key doesn't exist, create one

            dataDict[first] = {}



        setInDict(dataDict[first], rest, value)

    else:

        dataDict[first] = value

Pure Python style, without any import:

def nested_set(element, value, *keys):

    if type(element) is not dict:

        raise AttributeError('nested_set() expects dict as first argument.')

    if len(keys) < 2:

        raise AttributeError('nested_set() expects at least three arguments, not enough given.')



    _keys = keys[:-1]

    _element = element

    for key in _keys:

        _element = _element[key]

    _element[keys[-1]] = value



example = {"foo": { "bar": { "baz": "ok" } } }

keys = ['foo', 'bar']

nested_set(example, "yay", *keys)

print(example)

Output

{'foo': {'bar': 'yay'}}

An alternative way if you don't want to raise errors if one of the keys is absent (so that your main code can run without interruption):

def get_value(self,your_dict,*keys):

    curr_dict_ = your_dict

    for k in keys:

        v = curr_dict.get(k,None)

        if v is None:

            break

        if isinstance(v,dict):

            curr_dict = v

    return v

In this case, if any of the input keys is not present, None is returned, which can be used as a check in your main code to perform an alternative task.

How about check and then set dict element without processing all indexes twice?

Solution:

def nested_yield(nested, keys_list):

    """

    Get current nested data by send(None) method. Allows change it to Value by calling send(Value) next time

    :param nested: list or dict of lists or dicts

    :param keys_list: list of indexes/keys

    """

    if not len(keys_list):  # assign to 1st level list

        if isinstance(nested, list):

            while True:

                nested[:] = yield nested

        else:

            raise IndexError('Only lists can take element without key')





    last_key = keys_list.pop()

    for key in keys_list:

        nested = nested[key]



    while True:

        try:

            nested[last_key] = yield nested[last_key]

        except IndexError as e:

            print('no index {} in {}'.format(last_key, nested))

            yield None

Example workflow:

ny = nested_yield(nested_dict, nested_address)

data_element = ny.send(None)

if data_element:

    # process element

    ...

else:

    # extend/update nested data

    ny.send(new_data_element)

    ...

ny.close()

Test

>>> cfg= {'Options': [[1,[0]],[2,[4,[8,16]]],[3,[9]]]}

    ny = nested_yield(cfg, ['Options',1,1,1])

    ny.send(None)

[8, 16]

>>> ny.send('Hello!')

'Hello!'

>>> cfg

{'Options': [[1, [0]], [2, [4, 'Hello!']], [3, [9]]]}

>>> ny.close()

Solved this with recursion:

def get(d,l):

    if len(l)==1: return d[l[0]]

    return get(d[l[0]],l[1:])

Using your example:

dataDict = {

    "a":{

        "r": 1,

        "s": 2,

        "t": 3

        },

    "b":{

        "u": 1,

        "v": {

            "x": 1,

            "y": 2,

            "z": 3

        },

        "w": 3

        }

}

maplist1 = ["a", "r"]

maplist2 = ["b", "v", "y"]

print(get(dataDict, maplist1)) # 1

print(get(dataDict, maplist2)) # 2

If you also want the ability to work with arbitrary json including nested lists and dicts, and nicely handle invalid lookup paths, here's my solution:

from functools import reduce





def get_furthest(s, path):

    '''

    Gets the furthest value along a given key path in a subscriptable structure.



    subscriptable, list -> any

    :param s: the subscriptable structure to examine

    :param path: the lookup path to follow

    :return: a tuple of the value at the furthest valid key, and whether the full path is valid

    '''



    def step_key(acc, key):

        s = acc[0]

        if isinstance(s, str):

            return (s, False)

        try:

            return (s[key], acc[1])

        except LookupError:

            return (s, False)



    return reduce(step_key, path, (s, True))





def get_val(s, path):

    val, successful = get_furthest(s, path)

    if successful:

        return val

    else:

        raise LookupError('Invalid lookup path: {}'.format(path))





def set_val(s, path, value):

    get_val(s, path[:-1])[path[-1]] = value

It's satisfying to see these answers for having two static methods for setting & getting nested attributes. These solutions are way better than using nested trees https://gist.github.com/hrldcpr/2012250

Here's my implementation.

Usage:

To set nested attribute call sattr(my_dict, 1, 2, 3, 5) is equal to my_dict[1][2][3][4]=5

To get a nested attribute call gattr(my_dict, 1, 2)

def gattr(d, *attrs):

    """

    This method receives a dict and list of attributes to return the innermost value of the give dict       

    """

    try:

        for at in attrs:

            d = d[at]

        return d

    except:

        return None





def sattr(d, *attrs):

    """

    Adds "val" to dict in the hierarchy mentioned via *attrs

    For ex:

    sattr(animals, "cat", "leg","fingers", 4) is equivalent to animals["cat"]["leg"]["fingers"]=4

    This method creates necessary objects until it reaches the final depth

    This behaviour is also known as autovivification and plenty of implementation are around

    This implementation addresses the corner case of replacing existing primitives

    https://gist.github.com/hrldcpr/2012250#gistcomment-1779319

    """

    for attr in attrs[:-2]:

        # If such key is not found or the value is primitive supply an empty dict

        if d.get(attr) is None or isinstance(d.get(attr), dict):

            d[attr] = {}

        d = d[attr]

    d[attrs[-2]] = attrs[-1]

Very late to the party, but posting in case this may help someone in the future. For my use case, the following function worked the best. Works to pull any data type out of dictionary

dict is the dictionary containing our value

list is a list of "steps" towards our value

def getnestedvalue(dict, list):



    length = len(list)

    try:

        for depth, key in enumerate(list):

            if depth == length - 1:

                output = dict[key]

                return output

            dict = dict[key]

    except (KeyError, TypeError):

        return None



    return None

a method for concatenating strings:

def get_sub_object_from_path(dict_name, map_list):

    for i in map_list:

        _string = "['%s']" % i

        dict_name += _string

    value = eval(dict_name)

    return value

#Sample:

_dict = {'new': 'person', 'time': {'for': 'one'}}

map_list = ['time', 'for']

print get_sub_object_from_path("_dict",map_list)

#Output:

#one