llms-from-scratch-cn/Model_Architecture_Discussions/phi-3/phi-3.ipynb

In [1]:

%env HF_ENDPOINT=https://hf-mirror.com

env: HF_ENDPOINT=https://hf-mirror.com

In [2]:

from transformers import AutoTokenizer
from modeling_phi3 import Phi3ForCausalLM

model = Phi3ForCausalLM.from_pretrained("microsoft/phi-3-mini-4k-instruct")
tokenizer = AutoTokenizer.from_pretrained("microsoft/phi-3-mini-4k-instruct")

prompt = '\nDataWhalechina is an organization founded at Shanghai Jiao Tong University that helps learners learn artificial intelligence.'
inputs = tokenizer(prompt, return_tensors="pt")

# Generate
generate_ids = model.generate(inputs.input_ids, max_length=300)
tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]

Out [2]:

config.json:   0%|          | 0.00/476 [00:00<?, ?B/s]

model.safetensors.index.json:   0%|          | 0.00/832 [00:00<?, ?B/s]

Downloading shards:   0%|          | 0/2 [00:00<?, ?it/s]

model-00001-of-00002.safetensors:   0%|          | 0.00/4.97G [00:00<?, ?B/s]

model-00002-of-00002.safetensors:   0%|          | 0.00/2.67G [00:00<?, ?B/s]

Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]

generation_config.json:   0%|          | 0.00/172 [00:00<?, ?B/s]

tokenizer_config.json:   0%|          | 0.00/546 [00:00<?, ?B/s]

tokenizer.model:   0%|          | 0.00/500k [00:00<?, ?B/s]

tokenizer.json: 0.00B [00:00, ?B/s]

added_tokens.json:   0%|          | 0.00/293 [00:00<?, ?B/s]

special_tokens_map.json:   0%|          | 0.00/143 [00:00<?, ?B/s]

Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
You are not running the flash-attention implementation, expect numerical differences.

'\nDataWhalechina is an organization founded at Shanghai Jiao Tong University that helps learners learn artificial intelligence.\n'

In [5]:

prompt = '\nDataWhalechina is an organization founded at Shanghai Jiao Tong University that helps learners learn artificial '
inputs = tokenizer(prompt, return_tensors="pt")

# Generate
generate_ids = model.generate(inputs.input_ids, max_length=300)
tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]

Out [5]:

"\nDataWhalechina is an organization founded at Shanghai Jiao Tong University that helps learners learn artificial \nintelligence. We provide a variety of online courses for different skill levels and goals. Our courses are designed to be \nengaging, interactive, and effective, with a focus on practical application and real-world problem-solving. Whether you're \na beginner looking to get started in AI or an experienced professional looking to expand your skills, we have something \nfor everyone.\n\nOur courses cover a wide range of topics, including but not limited to:\n\n1. Introduction to Artificial Intelligence: Learn the basics of AI, including its history, key concepts, and real-world applications.\n2. Machine Learning: Explore the fundamentals of machine learning, including supervised and unsupervised learning, and popular \nalgorithms such as linear regression, decision trees, and neural networks.\n3. Deep Learning: Dive into the world of deep learning, including neural networks, convolutional neural networks (CNNs), and \nrecurrent neural networks (RNNs).\n4. Natural Language Processing (NLP): Learn how to build AI systems that can understand and generate human language, including \nsentiment analysis, language translation, and chatbots.\n5. Computer Vision: Discover how to teach computers"

Cell:

[Cell type raw - unsupported, skipped]