still confused how ceph crush maps work , hoping can shed light. here's osd tree:
core@store101 ~ $ ceph osd tree id weight type name up/down reweight primary-affinity -1 6.00000 root default -2 3.00000 datacenter dc1 -4 3.00000 rack rack_dc1 -10 1.00000 host store101 4 1.00000 osd.4 1.00000 1.00000 -7 1.00000 host store102 1 1.00000 osd.1 1.00000 1.00000 -9 1.00000 host store103 3 1.00000 osd.3 1.00000 1.00000 -3 3.00000 datacenter dc2 -5 3.00000 rack rack_dc2 -6 1.00000 host store104 0 1.00000 osd.0 1.00000 1.00000 -8 1.00000 host store105 2 1.00000 osd.2 1.00000 1.00000 -11 1.00000 host store106 5 1.00000 osd.5 1.00000 1.00000 i'm trying make sure that, replication value of 2 or more, replicas of object not in same datacenter. rule had (taken internet) is:
rule replicated_ruleset_dc { ruleset 0 type replicated min_size 1 max_size 10 step take default step choose firstn 2 type datacenter step choose firstn 2 type rack step chooseleaf firstn 0 type host step emit } however, if dump placement groups, straight off see 2 osd's same datacenter. osd's 5,0
core@store101 ~ $ ceph pg dump | grep 5,0 1.73 0 0 0 0 0 0 0 0 active+clean 2015-07-09 13:41:36.939197 0'0 96:113 [5,0] 5 [5,0] 5 0'0 2015-07-09 12:05:01.854945 0'0 2015-07-09 12:05:01.854945 1.70 0 0 0 0 0 0 0 0 active+clean 2015-07-09 13:41:36.947403 0'0 96:45 [5,0] 5 [5,0] 5 0'0 2015-07-09 12:05:01.854941 0'0 2015-07-09 12:05:01.854941 1.6f 0 0 0 0 0 0 0 0 active+clean 2015-07-09 13:41:36.947056 0'0 96:45 [5,0] 5 [5,0] 5 0'0 2015-07-09 12:05:01.854940 0'0 2015-07-09 12:05:01.854940 1.6c 0 0 0 0 0 0 0 0 active+clean 2015-07-09 13:41:36.938591 0'0 96:45 [5,0] 5 [5,0] 5 0'0 2015-07-09 12:05:01.854939 0'0 2015-07-09 12:05:01.854939 1.66 0 0 0 0 0 0 0 0 active+clean 2015-07-09 13:41:36.937803 0'0 96:107 [5,0] 5 [5,0] 5 0'0 2015-07-09 12:05:01.854936 0'0 2015-07-09 12:05:01.854936 1.67 0 0 0 0 0 0 0 0 active+clean 2015-07-09 13:41:36.929323 0'0 96:33 [5,0] 5 [5,0] 5 0'0 2015-07-09 12:05:01.854937 0'0 2015-07-09 12:05:01.854937 1.65 0 0 0 0 0 0 0 0 active+clean 2015-07-09 13:41:36.928200 0'0 96:33 [5,0] 5 [5,0] 5 0'0 2015-07-09 12:05:01.854936 0'0 2015-07-09 12:05:01.854936 1.63 0 0 0 0 0 0 0 0 active+clean 2015-07-09 13:41:36.927642 0'0 96:107 [5,0] 5 [5,0] 5 0'0 2015-07-09 12:05:01.854935 0'0 2015-07-09 12:05:01.854935 1.3f 0 0 0 0 0 0 0 0 active+clean 2015-07-09 13:41:36.924738 0'0 96:33 [5,0] 5 [5,0] 5 0'0 2015-07-09 12:05:01.854920 0'0 2015-07-09 12:05:01.854920 1.36 0 0 0 0 0 0 0 0 active+clean 2015-07-09 13:41:36.917833 0'0 96:45 [5,0] 5 [5,0] 5 0'0 2015-07-09 12:05:01.854916 0'0 2015-07-09 12:05:01.854916 1.33 0 0 0 0 0 0 0 0 active+clean 2015-07-09 13:41:36.911484 0'0 96:104 [5,0] 5 [5,0] 5 0'0 2015-07-09 12:05:01.854915 0'0 2015-07-09 12:05:01.854915 1.2b 0 0 0 0 0 0 0 0 active+clean 2015-07-09 13:41:36.878280 0'0 96:58 [5,0] 5 [5,0] 5 0'0 2015-07-09 12:05:01.854911 0'0 2015-07-09 12:05:01.854911 1.5 0 0 0 0 0 0 0 0 active+clean 2015-07-09 13:41:36.942620 0'0 96:98 [5,0] 5 [5,0] 5 0'0 2015-07-09 12:05:01.854892 0'0 2015-07-09 12:05:01.854892 how ensure @ least 1 replica alwasy in dc?
i changed ceph crush map yesterday: id weight type name up/down reweight primary-affinity -1 181.99979 root default -12 90.99989 rack rack1 -2 15.46999 host ceph0 1 3.64000 osd.1 1.00000 1.00000 0 3.64000 osd.0 1.00000 1.00000 8 2.73000 osd.8 1.00000 1.00000 9 2.73000 osd.9 1.00000 1.00000 19 2.73000 osd.19 1.00000 1.00000 ... -13 90.99989 rack rack2 -3 15.46999 host ceph2 2 3.64000 osd.2 1.00000 1.00000 3 3.64000 osd.3 1.00000 1.00000 10 2.73000 osd.10 1.00000 1.00000 11 2.73000 osd.11 1.00000 1.00000 18 2.73000 osd.18 1.00000 1.00000 ... rack rack1 { id -12 # not change unnecessarily # weight 91.000 alg straw hash 0 # rjenkins1 item ceph0 weight 15.470 ... } rack rack2 { id -13 # not change unnecessarily # weight 91.000 alg straw hash 0 # rjenkins1 item ceph2 weight 15.470 ... } root default { id -1 # not change unnecessarily # weight 182.000 alg straw hash 0 # rjenkins1 item rack1 weight 91.000 item rack2 weight 91.000 } rule racky { ruleset 3 type replicated min_size 1 max_size 10 step take default step chooseleaf firstn 0 type rack step emit } please show "root default" section
and try rule replicated_ruleset_dc { ruleset 0 type replicated min_size 1 max_size 10 step take default step chooseleaf firstn 0 type datacenter step emit }
Comments
Post a Comment