Weird thing to understand from mongoengine in Flask

Inital mongo collection

I am creation a mongo dument in Flask using MongoEngine Object mapper.Simple, consider the collection name is TestMe and it has two fields (name, address).
The name of the document is nava.

from mongoengine import Document, connect, StringField


class TestMe(Document):
    name = StringField(required=True)
    address = StringField(required=True)

Adding a new row

I am adding one row document in the collection using this mongoengine class which is described below.

 # 1. Initial document creation in mongo collections with only two fields(name and address)                                                        
    t = TestMe(name="Navaneethan", address="Bangalore")

So, the document is added successfully.

Verify in your mongo client

For quick confirmation, We can verify the document in mongod client.

(hire)navaneethan@navaneethLap:~/workspace/experiments/python/flask$ mongo
MongoDB shell version: 3.0.4
connecting to: test
Server has startup warnings: 
2015-07-30T10:19:08.103+0530 I CONTROL  [initandlisten] 
2015-07-30T10:19:08.103+0530 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2015-07-30T10:19:08.103+0530 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2015-07-30T10:19:08.103+0530 I CONTROL  [initandlisten] 
> show dbs;
employee  0.078GB
hirenew   0.078GB
local     0.078GB
nava      0.078GB
> use nava;
switched to db nava
> db.t
db.test_me          db.toLocaleString(  db.toString(        db.tojson(
> db.test_me.find()
{ "_id" : ObjectId("55b9bf94aff2d3212702a517"), "name" : "Navaneethan", "address" : "Bangalore" }

Adding a new field in the existing collection

Now I need a field has to be added in the existing db nava, which is is_active a booleanfield.
So, the refined model is,

class TestMe(Document):
    name = StringField(required=True)
    address = StringField(required=True)
    is_active = BooleanField(default=False)

To my current scenario, I have to update the field to the selected objects which would have the default value, but it should be reflected in db.

Updating the default value to the exisiting rows for the new field

in my python shell

obj = TestMe.objects(name='Navaneethan').first()
obj.is_active = False
print "updated successfully"

I am retrieving all the fields from the db and updating to the default value.The save happens successfully.

Verifying in the mongod client

But what surprising me is,
Unable to see the new field in mongo db.

switched to db nava
> db.test_me.find()
{ "_id" : ObjectId("55b9bf94aff2d3212702a517"), "name" : "Navaneethan", "address" : "Bangalore" }


Here I scrateched my head for almost 2 hours.

Changing the default value to some new value(false –> true)

After somepoint, tried to update the field with different value(not the default one)

obj = TestMe.objects(name='Navaneethan').first()
obj.is_active = True  # note here the default value of the field is changed
print "updated successfully"


Verifying in the mongod client

The field is updated to the existing rows.

> db.test_me.find()
{ "_id" : ObjectId("55b9bf94aff2d3212702a517"), "name" : "Navaneethan", "address" : "Bangalore", "is_active" : true }

my mind before hitting the head against wall:

It seems little weird to me. Since it is a nosql mapper or what? I come from django experience, I expected the field has to be updated even the default value is given to be updated for the existing rows. But it doesn’t happen. So, what happens is, If the value of the new field changed from the default value(`false` here) to the some other value(`true`) then the update reflects successfully.

There is a solution for every problem.

Yes. This is an obvious statement which I realized after a while hitting my head against the wall.

if you inherit from `DynamicDocument` for mongoengine odm. You will get the clear solution. It does the job.

The learn happens after this post gets published :)

Thanks for your time.

Posted in Flask, Python | Tagged , | Leave a comment

Erlang maps basics

erl> maps: for python>>>dict() people

just watch and think series:

How to create a new map in erlang?

 47> MyNewMap = maps:new().
 48> MyNewMap
 48> .
 49> maps:put(name, "Navaneethan", MyNewMap).
 #{name => "Navaneethan"}
 50> MyNewMap.
 51> MyNewMap2 = maps:put(name, "Navaneethan", MyNewMap).
 #{name => "Navaneethan"}
 52> MyNewMap2.
 #{name => "Navaneethan"}
 53> MyNewMap.
 54> maps:put(age, 26, MyNewMap2).
 #{age => 26,name => "Navaneethan"}
 55> MyNewMap2.
 #{name => "Navaneethan"}
 56> % RETURNS NEW MAP. Not updating into old one.%

Part2 in Maps(remove, put, update, without,values,to_list ):

 56> maps:remove('name',MyNewMap).
 57> maps:remove(name,MyNewMap).
 58> MyNewMap.
 59> maps:remove(name,MyNewMap2).
 60> MyNewMap2.
 #{name => "Navaneethan"}
 61> MyNewMap2 = maps:put(name, "Navaneethan", MyNewMap).
 #{name => "Navaneethan"}
 62> MyNewMap2 = maps:put(age, "26", MyNewMap).
 ** exception error: no match of right hand side value #{age => "26"}
 63> MyNewMap2.
 #{name => "Navaneethan"}
 64> f(MyNewMap2).
 65> MyNewMap2 = maps:put(age, "26", MyNewMap).
 #{age => "26"}
 66> MyNewMap2.
 #{age => "26"}
 67> MyNewMap2 = maps:put(work, "Python", MyNewMap2).
 ** exception error: no match of right hand side value #{age => "26",work => "Python"}
 68> MyNewMap3 = maps:put(work, "Python", MyNewMap2).
 #{age => "26",work => "Python"}
 69> maps:to_list(MyNewMap3).
 70> maps:update(work,"erlang",MyNewMap3).
 #{age => "26",work => "erlang"}
 71> MyNewMap3.
 #{age => "26",work => "Python"}
 72> %Update Returns New one.%
 72> maps:values(MyNewMap3).
 73> ReqKeys = [name, 'book', age, 'age'].
 74> maps:with(ReqKeys, MyNewMap3).
 ** exception error: undefined function maps:with/2
 75> MyNewMap3.
 #{age => "26",work => "Python"}
 76> maps:with([name], MyNewMap3).
 ** exception error: undefined function maps:with/2
 77> maps:with(ReqKeys, MyNewMap3).
 ** exception error: undefined function maps:with/2
 78> maps:without(ReqKeys, MyNewMap3).
 #{work => "Python"}

Points to be noticed:

This is erlang map which is similar to the python dictionary.Here we do some mandate operations with map.

If you pass your map to in built function in maps module, it returns new map which is unbounded.(line:62 & 67)

Rest of the things are similar like other programming languages.

Posted in erlang | Tagged | Leave a comment

Erlang day1

Erlang day1:

  1. Erlang shell an introduction
  2. How to use the erlang shell?
  3. few basic data structure tricks

Erlang shell an introduction:

In order to get your erlang shell,you have to install the erlang OTP for your platform(Windows/Linux).
Once installed erlang software, you can type erl to launch the shell.
In erl you can do the experiments in erlang.

watch the screen & read the below marks:

erl day 1

Few things keep remember:

The new values can’t be assigned to the bounded varialbes.Use always a unbound variable to store the value.Varialbes starts with caps(i.e Var).
_ is always used to unregister the value.The value will be omitted if it is assigned to the underscore.

Data structures are atoms, tuple, list.

  • tuple: It is represented by {}. e.g {1,2,3,4}.
  • atom: This is just a variable which is used name the data to match the pattern.
    eg.  {person, Name} = {person, "navaneethan"}.

    in this above example two tuples are matched with help of atom person, so the variable Name will be bounded to the value navaneethan.Finally, this pattern match is successful.

  • list: []
    e.g: [1,2,3,4].

Few weird things with tuples and lists:

  • Two tuples are compared with the no of elements on each tuple.
  • Two lists are compared with Dictionary associate property.
Posted in erlang | Tagged | 1 Comment

an intro to Erlang

an intro to Erlang:

I have started Erlang last weekend.Few things will amaze you if you start following it.You would get inspired of it’s powerful features.I would like to share with you my thoughts also what I learned in this one day.

Erlang creations:

Using erlang, there are quite # of powerful tools were built for real time usage.For example,

  • Facebook chat 100million requests are being handled
  • simpledb in AWS were implemented
  • Delicious bookmarking tool
  • sms component in TMobile
  • rabbitmq
  • couchdb
  • It has been tested in all sides in ATM AXD301 machine.

What erlang gives to us?

It is the concurrent programming language. It derives many features from various type of programming language to be focused o concurrent application implementation effectively.

So, Erlang guts are

  • No shared memory
  • Functional programming language
  • immutable and pattern matching

No sharing

There is nothing about shared memory in erlang virtual machine.The process communication is handled through message passing in asynchronous way to avoid synchronization overhead.So, it is the server side programming language, where you can derive the needful specific behavior(event handler, finite state machine) from erlang library to implement your server.

immutable and pattern matching

variables can’t be reassigned.The value of the variable can’t be changed.It has the feature called pattern matching to extract the data from the clause.

Functional programming language

Dynamic type system is used in erlang. Functional programming features are the core in erlang.

useful links


Posted in erlang | Tagged | 1 Comment

10min python: dis assembler

10min python: dis

what is dis?

dis is the disassembler. It shows to you how the python interpreter executes your python code in human readable format.

why do we need it?

To understand and optimise your python code to make it easier to the world.

how it works?

There is an inbuild module called dis which has the job and does it.It just reads the compiler understandable code(means byte code) and converting into mneumonics which is the cpu understandable language.

def your-function():



In [3]: import dis

In [4]: help(dis)

In [5]: def show_mybytes(a, b):
   ...:     l = []
   ...:     l.append(a)
   ...:     l.append(b)
   ...:     c = a + b
   ...:     l.append(c)
   ...:     return l

In [6]: show_mybytes(4,5)
Out[6]: [4, 5, 9]


In [8]: dis.dis(show_mybytes)
  2           0 BUILD_LIST               0
              3 STORE_FAST               2 (l)

  3           6 LOAD_FAST                2 (l)
              9 LOAD_ATTR                0 (append)
             12 LOAD_FAST                0 (a)
             15 CALL_FUNCTION            1
             18 POP_TOP             

  4          19 LOAD_FAST                2 (l)
             22 LOAD_ATTR                0 (append)
             25 LOAD_FAST                1 (b)
             28 CALL_FUNCTION            1
             31 POP_TOP             

  5          32 LOAD_FAST                0 (a)
             35 LOAD_FAST                1 (b)
             38 BINARY_ADD          
             39 STORE_FAST               3 (c)

  6          42 LOAD_FAST                2 (l)
             45 LOAD_ATTR                0 (append)
             48 LOAD_FAST                3 (c)
             51 CALL_FUNCTION            1
             54 POP_TOP             

  7          55 LOAD_FAST                2 (l)
             58 RETURN_VALUE        

In [9]: 

in this above example, first column is line no of our python code statements.Second column is the neumonic instraction address position(I think but not sure), third column is the mneumonic instructions for the cpu, last column is human understanble format mapping between python code to assembly code

Posted in 10min, Python | Tagged , , | Leave a comment

10min python: settrace

10min python:

what is settrace?

settrace traces each execution instruction of python script

Where we can use it?

If you want to trace the execution sequence and it’s control over your entire script, this should be the right utility.It does similar to Django’s middleware property

How to use it?

import sys

def callback(frame, event, arg):
    do your magic using frame, event, arg

#your script starts from here


import sys

def mycallback(frame, event, arg):
    print "event is %s" %event
    print "args are %s" %arg
    print "current exe line funcname: %s" %frame.f_code.co_name
    print "current execution line: %s" %frame.f_lineno
    print "caller line no: %s" %frame.f_back.f_lineno
    print "caller file name: %s" %frame.f_back.f_code.co_name
    return callback function is important to trace the sequence of all instraction from entry to exit, if return is `None` only one instruction will be traced
    return mycallback 

def function(arg1, arg2):
    print "Input: ",arg1, arg2
    return arg1 + arg2

print function(4,5)


settrace in pythonkudos links:

bye bye!


Posted in 10min, Academic, Python | Tagged , | Leave a comment

Knowledge is for Sharing

Knowledge is for sharing.

"Just even if you are the best programmer in the world, not sharing means then nothing more useful by you."
"The wealth of the tree is considered upon producing the fruits."

Here Shathimaan does his job. He made a book called i want 2 do project, tell me what 2 do which is the composed version of his experience.

I appreciate him first.

I ordered this book last week for Rs 399. I got it delivered by today. Initially, i felt little disappointed when i took first look of the book.Since, it has very less no of pages for the cost Rs 399.I started it, it went very well, everything was explained in crispy manner, no less in clarity. All flavors are mixed in the manner which will make strengthen the understanding of getting things done.

I realized in the midst of the chapter that I was wrong on my initial disappointment. 😦

I would say shortly, This book is like a “ABCD for the software programmers”. Everyone should have passed this understanding, before taking their weapons. I am happy at least I am getting understand right now. Thank you Shakthi Kannan for this wonderful piece. I am sure, this creature will make more creatures in open source industry.

Posted in Free Software, General, open source | Tagged , | 1 Comment