天天看點
傳回

可實作的全局唯一有序ID生成政策

在部落格園搜素全局唯一有序ID，羅列出來的文章大緻講述了以下幾個問題，常見的生成全局唯一id的常見方法 ：使用資料庫自動增長序列實作 ； 使用UUID實作；  使用redis實作； 使用Twitter的snowflake算法實作；使用資料庫+本地緩存實作。作為一個記錄性質的部落格，簡單總結一下。

在實際的生産場景中，經常會出現如下的情況比方說訂單号：D channelNo 流水号 樣例PSDK1600000001, PSDK1600000002, PSDK1600000003... 這種具有業務意義的全局唯一id且有序自增。先來看一下使用比較多的Twitter的snowflake算法，snowflake生成的ID整體上按照時間自增排序，并且整個分布式系統内不會産生ID碰撞（由datacenter和workerId作區分），并且效率較高。經測試snowflake每秒能夠産生26萬個ID。一個簡單的實作如下：

/**
 * Twitter的分布式自增ID雪花算法snowflake
 **/
public class SnowFlake {
    /**
     * 起始的時間戳
     */
    private final static long START_STMP = 1480166465631L;
    /**
     * 每一部分占用的位數
     */
    private final static long SEQUENCE_BIT = 12; //序列号占用的位數
    private final static long MACHINE_BIT = 5;   //機器辨別占用的位數
    private final static long DATACENTER_BIT = 5;//資料中心占用的位數
    /**
     * 每一部分的最大值
     */
    private final static long MAX_DATACENTER_NUM = -1L ^ (-1L << DATACENTER_BIT);
    private final static long MAX_MACHINE_NUM = -1L ^ (-1L << MACHINE_BIT);
    private final static long MAX_SEQUENCE = -1L ^ (-1L << SEQUENCE_BIT);
    /**
     * 每一部分向左的位移
     */
    private final static long MACHINE_LEFT = SEQUENCE_BIT;
    private final static long DATACENTER_LEFT = SEQUENCE_BIT + MACHINE_BIT;
    private final static long TIMESTMP_LEFT = DATACENTER_LEFT + DATACENTER_BIT;

    private long datacenterId;  //資料中心
    private long machineId;     //機器辨別
    private long sequence = 0L; //序列号
    private long lastStmp = -1L;//上一次時間戳

    public SnowFlake(long datacenterId, long machineId) {
        if (datacenterId > MAX_DATACENTER_NUM || datacenterId < 0) {
            throw new IllegalArgumentException("datacenterId can't be greater than MAX_DATACENTER_NUM or less than 0");
        }
        if (machineId > MAX_MACHINE_NUM || machineId < 0) {
            throw new IllegalArgumentException("machineId can't be greater than MAX_MACHINE_NUM or less than 0");
        }
        this.datacenterId = datacenterId;
        this.machineId = machineId;
    }

    /**
     * 産生下一個ID
     *
     * @return
     */
    public synchronized long nextId() {
        long currStmp = getNewstmp();
        if (currStmp < lastStmp) {
            throw new RuntimeException("Clock moved backwards.  Refusing to generate id");
        }

        if (currStmp == lastStmp) {
            //相同毫秒内，序列号自增
            sequence = (sequence + 1) & MAX_SEQUENCE;
            //同一毫秒的序列數已經達到最大
            if (sequence == 0L) {
                currStmp = getNextMill();
            }
        } else {
            //不同毫秒内，序列号置為0
            sequence = 0L;
        }

        lastStmp = currStmp;

        return (currStmp - START_STMP) << TIMESTMP_LEFT //時間戳部分
                | datacenterId << DATACENTER_LEFT       //資料中心部分
                | machineId << MACHINE_LEFT             //機器辨別部分
                | sequence;                             //序列号部分
    }

    private long getNextMill() {
        long mill = getNewstmp();
        while (mill <= lastStmp) {
            mill = getNewstmp();
        }
        return mill;
    }

    private long getNewstmp() {
        return System.currentTimeMillis();
    }

    public static void main(String[] args) {
        SnowFlake snowFlake = new SnowFlake(1, 1);
        long start = System.currentTimeMillis();
        for (int i = 0; i < 10; i++) {
            System.out.println(snowFlake.nextId());
        }
        System.out.println(System.currentTimeMillis() - start);
    }
}

算法中引入了時間因子，是以可以保證生成的id唯一且有序，但是滿足不了業務字段+流水号有序自增的要求。如果在此基礎上再配合使用資料庫本地緩存自然也是可以實作的，不過複雜化了。上述代碼執行兩次結果如下： 

385063405393940480
385063405393940481
385063405393940482
385063405393940483
385063405393940484
385063405393940485
385063405393940486
385063405393940487
385063405398134784
385063405398134785 

385064572152844288
385064572152844289
385064572152844290
385064572152844291
385064572152844292
385064572152844293
385064572152844294
385064572152844295
385064572152844296
385064572152844297

簡單的方法就是我們放棄自己造輪子的思想。mongodb中資料的基本單元稱為document，在一個特定集合内部需要唯一的辨別文檔，是以mongdb中存儲的文檔都由一個‘_id’鍵，這個鍵的值可以是任意類型的ObjectId，要求不同的機器都能用全局唯一的同種方法友善的生成它。是以不能使用自增主鍵，ObjectId 底層也是借鑒了雪花算法，使用12位元組的存儲空間  |0|1|2|3|4|5|6 |7|8|9|10|11|  |時間戳  |機器ID|PID|計數器|  前四個位元組時間戳是從标準紀元開始的時間戳，機關為秒 。時間戳保證秒級唯一，機器ID保證設計時考慮分布式，避免時鐘同步，PID保證同一台伺服器運作多個mongod執行個體時的唯一性，最後的計數器保證同一秒内的唯一性。

mongo在spring boot中的引入和配置，此處不再介紹。

建立model類

package com.slowcity.admin.generate.dbmodel;

import java.io.Serializable;

public class BaseSequence implements Serializable{

    private static final long serialVersionUID = 475722757687764546L;
    private String id;
    private String name;
    private Long sequence;

    public String getId() {
        return id;
    }
    public void setId(String id) {
        this.id = id;
    }
    public String getName() {
        return name;
    }
    public void setName(String name) {
        this.name = name;
    }
    public Long getSequence() {
        return sequence;
    }
    public void setSequence(Long sequence) {
        this.sequence = sequence;
    }
    @Override
    public int hashCode() {
        final int prime = 31;
        int result = 1;
        result = prime * result + ((id == null) ? 0 : id.hashCode());
        result = prime * result + ((name == null) ? 0 : name.hashCode());
        result = prime * result + ((sequence == null) ? 0 : sequence.hashCode());
        return result;
    }
    @Override
    public boolean equals(Object obj) {
        if (this == obj)
            return true;
        if (obj == null)
            return false;
        if (getClass() != obj.getClass())
            return false;
        BaseSequence other = (BaseSequence) obj;
        if (id == null) {
            if (other.id != null)
                return false;
        } else if (!id.equals(other.id))
            return false;
        if (name == null) {
            if (other.name != null)
                return false;
        } else if (!name.equals(other.name))
            return false;
        if (sequence == null) {
            if (other.sequence != null)
                return false;
        } else if (!sequence.equals(other.sequence))
            return false;
        return true;
    }
    @Override
    public String toString() {
        return "BaseSequence [id=" + id + ", name=" + name + ", sequence=" + sequence + "]";
    }

}      
public class DigitalTaskSequence extends BaseSequence{
    private static final long serialVersionUID = -7287622688931253780L;
}      
import org.springframework.data.annotation.Id;
import org.springframework.data.mongodb.core.mapping.Document;
import org.springframework.stereotype.Component;


@Component
@Document(collection = "dm_id_task")
public class DigitalTaskSequenceMG extends DigitalTaskSequence {
    private static final long serialVersionUID = -425011291271386371L;
    @Id
    @Override
    public String getId() {
        return super.getId();
    }
}

service

import java.util.List;

import com.slowcity.admin.generate.dbmodel.BaseSequence;

public interface SequenceGenericService {
    public String generateId(Class<? extends BaseSequence> clazz);
    List<BaseSequence> initAllId();
}      
import java.util.ArrayList;
import java.util.List;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import com.slowcity.admin.generate.dbmodel.BaseSequence;
import com.slowcity.admin.generate.dbmodel.DigitalTaskSequenceMG;
import com.slowcity.admin.generate.repository.SequenceGenericRepository;
import com.slowcity.admin.generate.service.SequenceGenericService;


@Service
@Transactional
public class SequenceGenericServiceImpl implements SequenceGenericService {
    private static final Logger log = LoggerFactory.getLogger(SequenceGenericServiceImpl.class);
    private SequenceGenericRepository sequenceGenericRepository;

    public SequenceGenericServiceImpl(SequenceGenericRepository sequenceGenericRepository) {
        this.sequenceGenericRepository = sequenceGenericRepository;
    }

    @Override
    public String generateId(Class<? extends BaseSequence> clazz) {
        String id = sequenceGenericRepository.generateId(clazz);
        log.info("{} generate {}", clazz.getName(), id);
        return id;
    }

    @Override
    public List<BaseSequence> initAllId() {

        List<BaseSequence> baseSequenceList = new ArrayList<>(),
        baseSequenceResultList = new ArrayList<>();
        DigitalTaskSequenceMG digitalTaskSequenceMG = new DigitalTaskSequenceMG();
        digitalTaskSequenceMG.setName("sequence");
        digitalTaskSequenceMG.setSequence(1210000000000000000L); //1210可以代表業務号 000000000000000代表自增流水号

        baseSequenceList.add(digitalTaskSequenceMG);
        
        for (BaseSequence baseSequence:baseSequenceList) {
            BaseSequence resultSequence = sequenceGenericRepository.initAllId(baseSequence);
            if(resultSequence != null){
                baseSequenceResultList.add(resultSequence);
            }
        }
        return baseSequenceResultList;
    }

   
}

資料實作層

import com.slowcity.admin.generate.dbmodel.BaseSequence;

public interface SequenceGenericRepository {

    public String generateId(Class<? extends BaseSequence> clazz);
    BaseSequence initAllId(BaseSequence Sequence);
    
}      
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;

import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.data.mongodb.core.query.Criteria;
import org.springframework.data.mongodb.core.query.Query;
import org.springframework.data.mongodb.core.query.Update;
import org.springframework.stereotype.Component;

import com.slowcity.admin.generate.dbmodel.BaseSequence;
import com.slowcity.admin.generate.repository.SequenceGenericRepository;

@Component
public class SequenceMongoGenericRepository implements SequenceGenericRepository {
    private Map<Class,Class<? extends BaseSequence>> baseSequenceMap;
    private MongoTemplate mongoTemplate;
    public SequenceMongoGenericRepository(List<BaseSequence> baseSequences, MongoTemplate mongoTemplate){
        baseSequenceMap = baseSequences.stream()
            .collect(Collectors.toMap(baseSequence -> baseSequence.getClass().getSuperclass(),
                BaseSequence::getClass));
        this.mongoTemplate = mongoTemplate;
    }


    @Override
    public String generateId(Class<? extends BaseSequence> clazz) {
        Class<? extends BaseSequence> childClazz = baseSequenceMap.get(clazz);
        if(childClazz != null) {
            Query query = new Query(Criteria.where("name").is("sequence"));
            Update update = new Update().inc("sequence", 1);
            Object dbm = mongoTemplate.findAndModify(query, update, childClazz);
            if(dbm != null) {
                BaseSequence bs = (BaseSequence)dbm;
                return String.valueOf(bs.getSequence());
            }
        }
        return null;
    }

    @Override
    public BaseSequence initAllId(BaseSequence Sequence) {

        Query query = new Query(Criteria.where("name").is("sequence"));
        Class clazz = Sequence.getClass();
        List<? extends BaseSequence> list = mongoTemplate.find(query,clazz);
        if(list.isEmpty()){
            mongoTemplate.save(Sequence);
            return Sequence;
        } 
            return null;
    }
}

controller

import java.util.List;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

import com.slowcity.admin.generate.dbmodel.BaseSequence;
import com.slowcity.admin.generate.dbmodel.DigitalTaskSequence;
import com.slowcity.admin.generate.service.SequenceGenericService;

/**
 * id生成器 
 * @author moona
 *
 */
@RestController
@RequestMapping("/generateId/task")
public class TaskGenerateIdController {

    @Autowired
    private SequenceGenericService sequenceGenericService;
    @RequestMapping(value = "/taskId", method = RequestMethod.GET)
    public String generateTaskId() {
        return sequenceGenericService.generateId(DigitalTaskSequence.class);
    }
    
    @RequestMapping(value = "/init", method = RequestMethod.GET)
    public  List<BaseSequence> generateTaskIdinit() {
        return sequenceGenericService.initAllId();
    }
    
}

執行初始化調用方法

可實作的全局唯一有序ID生成政策

 對應資料庫

可實作的全局唯一有序ID生成政策

開始測試生成id

第一次調用：

可實作的全局唯一有序ID生成政策

第2次調用

可實作的全局唯一有序ID生成政策

 第10次調用

可實作的全局唯一有序ID生成政策

 此時再檢視資料庫，序列已經到1210000000000000011 下次調用直接取值了。真正做到了了分布式滿足業務的自增全局唯一索引。mongo底層是原子性的，是以也不會出現并發的問題。如果将id生成政策部署成單台機器服務，則可以滿足不同服務不同業務的需求，真正做到可定制可擴充。盡可放心使用。

可實作的全局唯一有序ID生成政策

【end】

Mongo Id生成政策
上一篇: java循環控制語句loop使用
下一篇: Shell：執行腳本檔案方法

繼續閱讀