The total process can take awhile to setup Dolly. You’ll need a good internet connection and around 50GB of hard drive space.<\/p>\n\n\n\n
You’ll need to install the CUDA Toolkit to take advantage of the GPU. The GPU is much faster than just using the CPU.<\/p>\n\n\n\n
https:\/\/developer.nvidia.com\/cuda-downloads<\/a><\/p>\n\n\n\n Install git from the following site.<\/p>\n\n\n\n https:\/\/git-scm.com\/downloads<\/a><\/p>\n\n\n\n Download Dolly with git.<\/p>\n\n\n\n <\/p>\n\n\n\n We’ll also need Python installed if it is not already. Next we’ll need the following installed<\/p>\n\n\n\n The last one is needed to get Dolly to utilize a GPU.<\/p>\n\n\n\n Run a python console. If you run it as administrator, it should be faster.<\/p>\n\n\n\n Run the following commands to set up Dolly.<\/p>\n\n\n\n <\/p>\n\n\n\n Note: if you have issues, you may want\/need to specify an offload folder with offload_folder=”.\\offloadfolder”. An SSD is preferable. Alternatively, if we don’t want to trust_remote_code, we can do run the following<\/p>\n\n\n\n Now can ask Dolly a question.<\/p>\n\n\n\n Example:<\/p>\n\n\n\n <\/p>\n\n\n\n Further information is available at the following two links.<\/p>\n\n\n\n https:\/\/github.com\/databrickslabs\/dolly<\/a> The total process can take awhile to setup Dolly. You’ll need a good internet connection and around 50GB of hard drive space. Install Nvidia CUDA Toolkit You’ll need to install the CUDA Toolkit to take advantage of the GPU. The … Continue reading Install Git<\/h2>\n\n\n\n
Download Dolly<\/h2>\n\n\n\n
git lfs install \ngit clone https:\/\/huggingface.co\/databricks\/dolly-v2-12b<\/pre>\n\n\n\n
Install Python<\/h2>\n\n\n\n
https:\/\/www.python.org\/downloads\/release\/<\/a><\/p>\n\n\n\npy.exe -m pip install numpy<\/pre>\n\n\n\n
py.exe -m pip install accelerate>=0.12.0 transformers[torch]==4.25.1<\/pre>\n\n\n\n
py.exe -m pip install numpy --pre torch --force-reinstall --index-url https:\/\/download.pytorch.org\/whl\/nightly\/cu117 --user<\/pre>\n\n\n\n
Run Dolly<\/h2>\n\n\n\n
py.exe<\/pre>\n\n\n\n
import torch\nfrom transformers import pipeline\n\ngenerate_text = pipeline(model=\"databricks\/dolly-v2-3b\", torch_dtype=torch.bfloat16, trust_remote_code=True, device_map=\"auto\")\n\n# Or to use the full model run\n\ngenerate_text = pipeline(model=\"databricks\/dolly-v2-12b\", torch_dtype=torch.bfloat16, trust_remote_code=True, device_map=\"auto\")<\/pre>\n\n\n\n
Also if you have lots of RAM, you can take out the “torch_dtype=torch.bfloat16”<\/p>\n\n\n\nfrom instruct_pipeline import InstructionTextGenerationPipeline\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\ntokenizer = AutoTokenizer.from_pretrained(\"databricks\/dolly-v2-3b\", padding_side=\"left\")\nmodel = AutoModelForCausalLM.from_pretrained(\"databricks\/dolly-v2-3b\", device_map=\"auto\")\n\ngenerate_text = InstructionTextGenerationPipeline(model=model, tokenizer=tokenizer)<\/pre>\n\n\n\n
generate_text(\"Your question?\")<\/pre>\n\n\n\n
>>> generate_text(\"Tell me about Databricks dolly-v2-3b?\")\n'Dolly is the fully managed open-source engine that allows you to rapidly build, test, and deploy machine learning models, all on your own infrastructure.'<\/pre>\n\n\n\n
https:\/\/huggingface.co\/databricks\/dolly-v2-3b<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"